[jira] [Created] (HADOOP-18163) hadoop-azure support for the Manifest Committer of MAPREDUCE-7341

2022-03-16 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18163: --- Summary: hadoop-azure support for the Manifest Committer of MAPREDUCE-7341 Key: HADOOP-18163 URL: https://issues.apache.org/jira/browse/HADOOP-18163 Project: Ha

[jira] [Created] (HADOOP-18162) Hadoop common enhancements for the Manifest Committer ofMAPREDUCE-7341

2022-03-16 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18162: --- Summary: Hadoop common enhancements for the Manifest Committer ofMAPREDUCE-7341 Key: HADOOP-18162 URL: https://issues.apache.org/jira/browse/HADOOP-18162 Projec

[jira] [Resolved] (HADOOP-18157) S3AFileSystem cannot find a fake dir any more

2022-03-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18157. - Resolution: Duplicate > S3AFileSystem cannot find a fake dir any more > ---

[jira] [Resolved] (HADOOP-18150) Fix ITestAuditManagerDisabled after S3A audit logging was enabled in HADOOP-18091

2022-03-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18150. - Fix Version/s: 3.3.3 Resolution: Fixed > Fix ITestAuditManagerDisabled after S3A

[jira] [Resolved] (HADOOP-18075) ABFS: Fix failure caused by listFiles() in ITestAbfsRestOperationException

2022-03-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18075. - Fix Version/s: 3.3.3 Resolution: Fixed > ABFS: Fix failure caused by listFiles()

[jira] [Resolved] (HADOOP-17862) ABFS: Fix unchecked cast compiler warning for AbfsListStatusRemoteIterator

2022-03-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17862. - Fix Version/s: 3.3.3 Resolution: Fixed > ABFS: Fix unchecked cast compiler warni

[jira] [Resolved] (HADOOP-17765) ABFS: Use Unique File Paths in Tests

2022-03-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17765. - Fix Version/s: 3.3.3 Resolution: Fixed > ABFS: Use Unique File Paths in Tests >

[jira] [Resolved] (HADOOP-18147) ITestAbfsRestOperationException failing

2022-02-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18147. - Resolution: Duplicate > ITestAbfsRestOperationException failing > -

[jira] [Created] (HADOOP-18147) ITestAbfsRestOperationException failing

2022-02-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18147: --- Summary: ITestAbfsRestOperationException failing Key: HADOOP-18147 URL: https://issues.apache.org/jira/browse/HADOOP-18147 Project: Hadoop Common Issue

[jira] [Resolved] (HADOOP-18071) ABFS: Set driver global timeout for ITestAzureBlobFileSystemBasics

2022-02-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18071. - Fix Version/s: 3.3.3 Resolution: Fixed > ABFS: Set driver global timeout for ITe

[jira] [Created] (HADOOP-18137) Verify FileUtils.unTar() handling of missing .tgz files

2022-02-21 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18137: --- Summary: Verify FileUtils.unTar() handling of missing .tgz files Key: HADOOP-18137 URL: https://issues.apache.org/jira/browse/HADOOP-18137 Project: Hadoop Common

[jira] [Created] (HADOOP-18136) Verify FileUtils.unTar() handling of missing .tar files

2022-02-21 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18136: --- Summary: Verify FileUtils.unTar() handling of missing .tar files Key: HADOOP-18136 URL: https://issues.apache.org/jira/browse/HADOOP-18136 Project: Hadoop Common

[jira] [Resolved] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-02-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18091. - Fix Version/s: 3.3.3 Resolution: Fixed > S3A auditing leaks memory through Threa

[jira] [Resolved] (HADOOP-18114) Documentation Syntax Error Fix > AWS Assumed Roles

2022-02-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18114. - Fix Version/s: 3.3.3 Resolution: Fixed thanks! > Documentation Syntax Error Fix

[jira] [Resolved] (HADOOP-18085) S3 SDK Upgrade causes AccessPoint ARN endpoint mistranslation

2022-02-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18085. - Fix Version/s: 3.3.3 Resolution: Fixed this in branch-3 now, so all branches wit

[jira] [Created] (HADOOP-18113) remove filtering of directory markers in s3a RenameOperation

2022-02-03 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18113: --- Summary: remove filtering of directory markers in s3a RenameOperation Key: HADOOP-18113 URL: https://issues.apache.org/jira/browse/HADOOP-18113 Project: Hadoop

[jira] [Created] (HADOOP-18095) s3a connector to fully support AWS partitions,

2022-01-24 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18095: --- Summary: s3a connector to fully support AWS partitions, Key: HADOOP-18095 URL: https://issues.apache.org/jira/browse/HADOOP-18095 Project: Hadoop Common

[jira] [Resolved] (HADOOP-18094) Disable S3A auditing by default.

2022-01-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18094. - Fix Version/s: 3.3.2 Resolution: Fixed > Disable S3A auditing by default. >

[jira] [Created] (HADOOP-18094) Disable S3A auditing by default.

2022-01-24 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18094: --- Summary: Disable S3A auditing by default. Key: HADOOP-18094 URL: https://issues.apache.org/jira/browse/HADOOP-18094 Project: Hadoop Common Issue Type:

[jira] [Created] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

2022-01-22 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18091: --- Summary: S3A auditing leaks memory through ThreadLocal references Key: HADOOP-18091 URL: https://issues.apache.org/jira/browse/HADOOP-18091 Project: Hadoop Commo

[jira] [Resolved] (HADOOP-18084) ABFS: Add testfilePath while verifying test contents are read correctly

2022-01-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18084. - Fix Version/s: 3.3.3 Resolution: Fixed > ABFS: Add testfilePath while verifying

[jira] [Resolved] (HADOOP-17409) Remove S3Guard - no longer needed

2022-01-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17409. - Fix Version/s: 3.3.3 (was: 3.4.0) Resolution: Fixed > Rem

[jira] [Resolved] (HADOOP-17457) intermittent ITestS3AInconsistency.testGetFileStatus failure.

2022-01-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17457. - Fix Version/s: 3.3.3 Resolution: Duplicate > intermittent ITestS3AInconsistency.

[jira] [Resolved] (HADOOP-18068) upgrade AWS SDK to 1.12.132

2022-01-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18068. - Fix Version/s: 3.3.3 Resolution: Fixed TestArn from HADOOP-17198 broke with this

[jira] [Created] (HADOOP-18081) FileNotFoundException in abfs mkdirs() call

2022-01-13 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18081: --- Summary: FileNotFoundException in abfs mkdirs() call Key: HADOOP-18081 URL: https://issues.apache.org/jira/browse/HADOOP-18081 Project: Hadoop Common I

[jira] [Resolved] (HADOOP-17954) org.apache.spark.SparkException: Task failed while writing rows S3

2022-01-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17954. - Resolution: Cannot Reproduce > org.apache.spark.SparkException: Task failed while writi

[jira] [Resolved] (HADOOP-14334) S3 SSEC tests to downgrade when running against a mandatory encryption object store

2022-01-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14334. - Fix Version/s: 3.3.3 Resolution: Fixed > S3 SSEC tests to downgrade when runnin

[jira] [Resolved] (HADOOP-17736) Über-JIRA: abfs phase III: Hadoop 3.3.2 features & fixes

2022-01-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17736. - Resolution: Done > Über-JIRA: abfs phase III: Hadoop 3.3.2 features & fixes > -

[jira] [Resolved] (HADOOP-17492) abfs listLocatedStatus to support incremental/async page fetching

2022-01-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17492. - Fix Version/s: 3.3.3 Resolution: Fixed done in HADOOP-17979 so etags are returne

[jira] [Created] (HADOOP-18072) Über-JIRA: abfs phase III: Hadoop 3.3.3 features & fixes

2022-01-07 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18072: --- Summary: Über-JIRA: abfs phase III: Hadoop 3.3.3 features & fixes Key: HADOOP-18072 URL: https://issues.apache.org/jira/browse/HADOOP-18072 Project: Hadoop Commo

[jira] [Created] (HADOOP-18068) upgrade AWS SDK to 1.12.x

2022-01-05 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18068: --- Summary: upgrade AWS SDK to 1.12.x Key: HADOOP-18068 URL: https://issues.apache.org/jira/browse/HADOOP-18068 Project: Hadoop Common Issue Type: Sub-tas

[jira] [Resolved] (HADOOP-17566) Über-jira: S3A Hadoop 3.3.2 features

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17566. - Fix Version/s: 3.3.2 Resolution: Done > Über-jira: S3A Hadoop 3.3.2 features > -

[jira] [Resolved] (HADOOP-15751) AWS Data read stack trace in S3a putObjectDirect

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15751. - Resolution: Cannot Reproduce not seen for a long time; closing. re-open if it surfaces

[jira] [Resolved] (HADOOP-15955) declare that fs.s3a.ext. is a prefix for arbitrary extensions

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15955. - Fix Version/s: 3.3.2 Resolution: Done > declare that fs.s3a.ext. is a prefix for

[jira] [Resolved] (HADOOP-16370) S3AFileSystem copyFile to propagate etag/version from getObjectMetadata to copy request

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16370. - Resolution: Duplicate > S3AFileSystem copyFile to propagate etag/version from getObject

[jira] [Resolved] (HADOOP-14070) S3a: Failed to reset the request input stream/make S3A uploadPart() retriable

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14070. - Resolution: Cannot Reproduce not seen for a long time; closing. re-open if it surfaces

[jira] [Resolved] (HADOOP-16188) s3a rename failed during copy, "Unable to copy part" + 200 error code

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16188. - Resolution: Cannot Reproduce > s3a rename failed during copy, "Unable to copy part" + 2

[jira] [Resolved] (HADOOP-17226) Failure of ITestAssumeRole.testRestrictedCommitActions

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17226. - Fix Version/s: 3.3.2 Resolution: Fixed > Failure of ITestAssumeRole.testRestrict

[jira] [Resolved] (HADOOP-17263) transient ITestS3AFileContextStatistics failure -read buffer not filled

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17263. - Resolution: Won't Fix usual transient buffer underrun issue > transient ITestS3AFileCo

[jira] [Resolved] (HADOOP-17403) S3A ITestPartialRenamesDeletes.testRenameDirFailsInDelete failure: missing directory marker

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17403. - Resolution: Won't Fix remove s3guard and this whole feature gets cut, so no test failur

[jira] [Resolved] (HADOOP-17818) ITestS3AContractDistCp failing IllegalStateException: WFOPENSSL0030 Running handshake with buffered unwrapped data

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17818. - Resolution: Cannot Reproduce > ITestS3AContractDistCp failing IllegalStateException: WF

[jira] [Created] (HADOOP-18067) Über-jira: S3A Hadoop 3.3.3 features

2022-01-05 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18067: --- Summary: Über-jira: S3A Hadoop 3.3.3 features Key: HADOOP-18067 URL: https://issues.apache.org/jira/browse/HADOOP-18067 Project: Hadoop Common Issue Ty

[jira] [Resolved] (HADOOP-18064) replace URL.toString() with URL.toURI().toString()

2022-01-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18064. - Resolution: Invalid > replace URL.toString() with URL.toURI().toString() >

[jira] [Created] (HADOOP-18064) replace URL.toString() with URL.toURI().toString()

2022-01-04 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18064: --- Summary: replace URL.toString() with URL.toURI().toString() Key: HADOOP-18064 URL: https://issues.apache.org/jira/browse/HADOOP-18064 Project: Hadoop Common

[jira] [Resolved] (HADOOP-17989) ITestAzureBlobFileSystemDelete failing "Operations has null HTTP response"

2021-12-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17989. - Fix Version/s: 3.3.3 Resolution: Fixed > ITestAzureBlobFileSystemDelete failing

[jira] [Resolved] (HADOOP-18026) Fix default value of Magic committer

2021-12-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18026. - Fix Version/s: 3.3.3 Resolution: Fixed > Fix default value of Magic committer >

[jira] [Resolved] (HADOOP-18021) Provide a public wrapper of Configuration#substituteVars

2021-12-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18021. - Resolution: Fixed > Provide a public wrapper of Configuration#substituteVars >

[jira] [Resolved] (HADOOP-17979) Interface EtagSource to allow FileStatus subclasses to provide etags

2021-11-29 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17979. - Fix Version/s: 3.3.2 Resolution: Fixed > Interface EtagSource to allow FileStatu

[jira] [Created] (HADOOP-18013) ABFS: add resilient trash policy

2021-11-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18013: --- Summary: ABFS: add resilient trash policy Key: HADOOP-18013 URL: https://issues.apache.org/jira/browse/HADOOP-18013 Project: Hadoop Common Issue Type:

[jira] [Created] (HADOOP-18004) abfs and s3a disk buffer factories to use UUIDs for file prefixes

2021-11-11 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18004: --- Summary: abfs and s3a disk buffer factories to use UUIDs for file prefixes Key: HADOOP-18004 URL: https://issues.apache.org/jira/browse/HADOOP-18004 Project: Ha

[jira] [Resolved] (HADOOP-17981) Support etag-assisted renames in FileOutputCommitter

2021-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17981. - Resolution: Won't Fix closing as WONTFIX as even though the PR is complete, we don't wa

[jira] [Created] (HADOOP-18002) abfs

2021-11-10 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18002: --- Summary: abfs Key: HADOOP-18002 URL: https://issues.apache.org/jira/browse/HADOOP-18002 Project: Hadoop Common Issue Type: Bug Reporter: St

[jira] [Resolved] (HADOOP-17928) s3a: set fs.s3a.downgrade.syncable.exceptions = true by default

2021-11-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17928. - Fix Version/s: 3.3.2 Resolution: Fixed > s3a: set fs.s3a.downgrade.syncable.exce

[jira] [Reopened] (HADOOP-17873) ABFS: Fix transient failures in ITestAbfsStreamStatistics and ITestAbfsRestOperationException

2021-11-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-17873: - had to reopen it as now build rules keep the shaded guava @VislbleForTesting refs out; you

[jira] [Resolved] (HADOOP-17863) ABFS: Fix compiler deprecation warning in TextFileBasedIdentityHandler

2021-11-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17863. - Fix Version/s: 3.3.2 Resolution: Fixed > ABFS: Fix compiler deprecation warning

[jira] [Resolved] (HADOOP-17873) ABFS: Fix transient failures in ITestAbfsStreamStatistics and ITestAbfsRestOperationException

2021-11-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17873. - Fix Version/s: 3.4.0 Resolution: Fixed > ABFS: Fix transient failures in ITestAb

[jira] [Resolved] (HADOOP-17966) S3A SSE-KMS inconsistency issue during rename

2021-11-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17966. - Resolution: Duplicate > S3A SSE-KMS inconsistency issue during rename > ---

[jira] [Created] (HADOOP-17989) ITestAzureBlobFileSystemDelete failing "Operations has null HTTP response"

2021-11-04 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17989: --- Summary: ITestAzureBlobFileSystemDelete failing "Operations has null HTTP response" Key: HADOOP-17989 URL: https://issues.apache.org/jira/browse/HADOOP-17989 Pr

[jira] [Resolved] (HADOOP-17945) JsonSerialization raises EOFException reading JSON data stored on google GCS

2021-11-02 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17945. - Fix Version/s: (was: 3.4.0) 3.3.2 Resolution: Fixed > Jso

[jira] [Resolved] (HADOOP-17984) Hadoop-aws jar is unable to read file from S3 if used with third party like MINIO

2021-10-29 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17984. - Resolution: Invalid > Hadoop-aws jar is unable to read file from S3 if used with third

[jira] [Created] (HADOOP-17981) Support etag-assisted renames in FileOutputCommitter

2021-10-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17981: --- Summary: Support etag-assisted renames in FileOutputCommitter Key: HADOOP-17981 URL: https://issues.apache.org/jira/browse/HADOOP-17981 Project: Hadoop Common

[jira] [Resolved] (HADOOP-17980) Spark application stuck at ACCEPTED state (unset port issue)

2021-10-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17980. - Resolution: Invalid > Spark application stuck at ACCEPTED state (unset port issue) > --

[jira] [Created] (HADOOP-17979) Interface EtagFromFileStatus to allow FileStatus subclasses to provide etags

2021-10-26 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17979: --- Summary: Interface EtagFromFileStatus to allow FileStatus subclasses to provide etags Key: HADOOP-17979 URL: https://issues.apache.org/jira/browse/HADOOP-17979

[jira] [Created] (HADOOP-17976) abfs etag extraction inconsistent between LIST and HEAD calls

2021-10-22 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17976: --- Summary: abfs etag extraction inconsistent between LIST and HEAD calls Key: HADOOP-17976 URL: https://issues.apache.org/jira/browse/HADOOP-17976 Project: Hadoop

[jira] [Resolved] (HADOOP-17974) Fix the import statements in hadoop-aws module

2021-10-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17974. - Fix Version/s: 3.3.2 Resolution: Fixed merged trunk & branch-3.3; accidentally l

[jira] [Resolved] (HADOOP-17953) S3A: ITestS3AFileContextStatistics test to lookup global or per-bucket configuration for encryption algorithm

2021-10-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17953. - Fix Version/s: 3.4.0 Resolution: Fixed > S3A: ITestS3AFileContextStatistics test

[jira] [Resolved] (HADOOP-16646) Backport S3A enhancements and fixes from trunk to branch-3.2

2021-10-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16646. - Resolution: Won't Fix > Backport S3A enhancements and fixes from trunk to branch-3.2 >

[jira] [Resolved] (HADOOP-17559) S3Guard import can OOM on large imports

2021-10-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17559. - Resolution: Won't Fix > S3Guard import can OOM on large imports > -

[jira] [Created] (HADOOP-17961) s3 and abfs incremental listing: ask for a smaller first batch

2021-10-11 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17961: --- Summary: s3 and abfs incremental listing: ask for a smaller first batch Key: HADOOP-17961 URL: https://issues.apache.org/jira/browse/HADOOP-17961 Project: Hadoo

[jira] [Resolved] (HADOOP-17951) AccessPoint verifyBucketExistsV2 always returns false

2021-10-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17951. - Fix Version/s: 3.4.0 Resolution: Fixed > AccessPoint verifyBucketExistsV2 always

[jira] [Resolved] (HADOOP-17198) Support S3 Access Points

2021-10-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17198. - Resolution: Fixed in trunk, will get into 3.3.x once the CSE stuff is in > Support S3

[jira] [Resolved] (HADOOP-17934) NullPointerException when no HTTP response set on AbfsRestOperation

2021-09-30 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17934. - Fix Version/s: 3.3.3 Resolution: Fixed > NullPointerException when no HTTP respo

[jira] [Resolved] (HADOOP-17520) S3A to downgraded to unguarded if the DDB table cannot be found

2021-09-30 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17520. - Resolution: Won't Fix we have the PR but I'm not going to merge it > S3A to downgraded

[jira] [Resolved] (HADOOP-17851) S3A to support user-specified content encoding

2021-09-29 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17851. - Fix Version/s: 3.4.0 Resolution: Fixed > S3A to support user-specified content e

[jira] [Created] (HADOOP-17945) JsonSerialization raises EOFException reading JSON off google GCS

2021-09-29 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17945: --- Summary: JsonSerialization raises EOFException reading JSON off google GCS Key: HADOOP-17945 URL: https://issues.apache.org/jira/browse/HADOOP-17945 Project: Ha

[jira] [Created] (HADOOP-17943) Add s3a tool to convert S3 server logs to avro/csv files

2021-09-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17943: --- Summary: Add s3a tool to convert S3 server logs to avro/csv files Key: HADOOP-17943 URL: https://issues.apache.org/jira/browse/HADOOP-17943 Project: Hadoop Commo

[jira] [Created] (HADOOP-17942) abfs & s3a FS instantiate triggers warning about deprecated io.bytes.per.checksum

2021-09-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17942: --- Summary: abfs & s3a FS instantiate triggers warning about deprecated io.bytes.per.checksum Key: HADOOP-17942 URL: https://issues.apache.org/jira/browse/HADOOP-17942

[jira] [Resolved] (HADOOP-7199) Crash due to reuse of checksum files

2021-09-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-7199. Resolution: Not A Problem > Crash due to reuse of checksum files > ---

[jira] [Created] (HADOOP-17937) .ITestS3ADeleteFilesOneByOne. testBulkRenameAndDelete OOM: Direct buffer memory

2021-09-24 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17937: --- Summary: .ITestS3ADeleteFilesOneByOne. testBulkRenameAndDelete OOM: Direct buffer memory Key: HADOOP-17937 URL: https://issues.apache.org/jira/browse/HADOOP-17937

[jira] [Resolved] (HADOOP-17195) Intermittent OutOfMemory error while performing hdfs CopyFromLocal to abfs

2021-09-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17195. - Fix Version/s: 3.3.3 Resolution: Fixed > Intermittent OutOfMemory error while pe

[jira] [Created] (HADOOP-17928) s3a: set fs.s3a.downgrade.syncable.exceptions = true by default

2021-09-21 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17928: --- Summary: s3a: set fs.s3a.downgrade.syncable.exceptions = true by default Key: HADOOP-17928 URL: https://issues.apache.org/jira/browse/HADOOP-17928 Project: Hado

[jira] [Resolved] (HADOOP-13637) improve setting of max connections in AWS client

2021-09-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13637. - Fix Version/s: 3.3.2 Resolution: Duplicate > improve setting of max connections

[jira] [Created] (HADOOP-17915) ABFS AbfsDelegationTokenManager to generate canonicalServiceName if DT plugin doesn't

2021-09-15 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17915: --- Summary: ABFS AbfsDelegationTokenManager to generate canonicalServiceName if DT plugin doesn't Key: HADOOP-17915 URL: https://issues.apache.org/jira/browse/HADOOP-17915

[jira] [Resolved] (HADOOP-17894) CredentialProviderFactory.getProviders() recursion loading JCEKS file from s3a

2021-09-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17894. - Fix Version/s: 3.3.2 Resolution: Fixed > CredentialProviderFactory.getProviders(

[jira] [Created] (HADOOP-17906) AbstractAbfsIntegrationTest.createFilesystemForSASTests() leaks FS instances

2021-09-09 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17906: --- Summary: AbstractAbfsIntegrationTest.createFilesystemForSASTests() leaks FS instances Key: HADOOP-17906 URL: https://issues.apache.org/jira/browse/HADOOP-17906

[jira] [Resolved] (HADOOP-17618) ABFS: Partially obfuscate SAS object IDs in Logs

2021-09-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17618. - Fix Version/s: 3.3.2 Resolution: Fixed > ABFS: Partially obfuscate SAS object ID

[jira] [Resolved] (HADOOP-17898) Upgrade BouncyCastle to 1.69

2021-09-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17898. - Resolution: Duplicate I'm closing this as a duplicate...put the PRs under HADOOP-17563

[jira] [Created] (HADOOP-17894) CredentialProviderFactory.getProviders() to detect recursion

2021-09-06 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17894: --- Summary: CredentialProviderFactory.getProviders() to detect recursion Key: HADOOP-17894 URL: https://issues.apache.org/jira/browse/HADOOP-17894 Project: Hadoop

[jira] [Created] (HADOOP-17884) FsShell to use openFile(sequential) for reads

2021-09-01 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17884: --- Summary: FsShell to use openFile(sequential) for reads Key: HADOOP-17884 URL: https://issues.apache.org/jira/browse/HADOOP-17884 Project: Hadoop Common

[jira] [Created] (HADOOP-17883) openFile() to add standard options for read policies, start/end ranges

2021-09-01 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17883: --- Summary: openFile() to add standard options for read policies, start/end ranges Key: HADOOP-17883 URL: https://issues.apache.org/jira/browse/HADOOP-17883 Projec

[jira] [Created] (HADOOP-17882) distcp to use openFile() with sequential IO; ranges of reads

2021-09-01 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17882: --- Summary: distcp to use openFile() with sequential IO; ranges of reads Key: HADOOP-17882 URL: https://issues.apache.org/jira/browse/HADOOP-17882 Project: Hadoop

[jira] [Created] (HADOOP-17881) S3A DeleteOperation to parallelize POSTing of bulk deletes

2021-09-01 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17881: --- Summary: S3A DeleteOperation to parallelize POSTing of bulk deletes Key: HADOOP-17881 URL: https://issues.apache.org/jira/browse/HADOOP-17881 Project: Hadoop Com

[jira] [Resolved] (HADOOP-17869) fs.s3a.connection.maximum should be bigger than fs.s3a.threads.max

2021-08-30 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17869. - Fix Version/s: 3.3.2 Resolution: Fixed > fs.s3a.connection.maximum should be big

[jira] [Created] (HADOOP-17876) ABFS to log @ debug each config option *And where it comes from*

2021-08-27 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17876: --- Summary: ABFS to log @ debug each config option *And where it comes from* Key: HADOOP-17876 URL: https://issues.apache.org/jira/browse/HADOOP-17876 Project: Had

[jira] [Resolved] (HADOOP-17677) Distcp is unable to determine region with S3 PrivateLink endpoints

2021-08-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17677. - Fix Version/s: 3.3.2 Resolution: Fixed > Distcp is unable to determine region wi

[jira] [Created] (HADOOP-17871) S3A CSE: minor tuning

2021-08-26 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17871: --- Summary: S3A CSE: minor tuning Key: HADOOP-17871 URL: https://issues.apache.org/jira/browse/HADOOP-17871 Project: Hadoop Common Issue Type: Sub-task

[jira] [Created] (HADOOP-17856) JsonSerDeser to collect IOStatistics

2021-08-19 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17856: --- Summary: JsonSerDeser to collect IOStatistics Key: HADOOP-17856 URL: https://issues.apache.org/jira/browse/HADOOP-17856 Project: Hadoop Common Issue Ty

[jira] [Resolved] (HADOOP-17836) ABFS connection reset reporting excessively noisy

2021-08-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17836. - Fix Version/s: 3.4.0 Resolution: Fixed > ABFS connection reset reporting excessi

[jira] [Resolved] (HADOOP-17842) HADOOP-AWS with Spark on Kubernetes (EKS)

2021-08-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17842. - Resolution: Works for Me > HADOOP-AWS with Spark on Kubernetes (EKS) >

[jira] [Created] (HADOOP-17839) LocalFS to support ability to disable permission get/set; remove need for winutils

2021-08-06 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17839: --- Summary: LocalFS to support ability to disable permission get/set; remove need for winutils Key: HADOOP-17839 URL: https://issues.apache.org/jira/browse/HADOOP-17839

<    2   3   4   5   6   7   8   9   10   11   >