[jira] [Created] (HADOOP-17400) Optimize S3A for maximum performance in directory listings

2020-11-30 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17400: --- Summary: Optimize S3A for maximum performance in directory listings Key: HADOOP-17400 URL: https://issues.apache.org/jira/browse/HADOOP-17400 Project: Hadoop

[jira] [Resolved] (HADOOP-15844) tag S3GuardTool entry points as limitedPrivate("management-tools")/evolving

2020-11-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15844. - Fix Version/s: 3.3.1 Resolution: Fixed Came in with HADOOP-13280 directory

[jira] [Resolved] (HADOOP-16382) Clock skew can cause S3Guard to think object metadata is out of date

2020-11-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16382. - Resolution: Won't Fix we just don't get enough information back. Be nice if an http

[jira] [Resolved] (HADOOP-17398) Skipping network I/O in S3A getFileStatus(/) breaks some tests

2020-11-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17398. - Fix Version/s: 3.3.1 Resolution: Fixed > Skipping network I/O in S3A

[jira] [Resolved] (HADOOP-17385) ITestS3ADeleteCost.testDirMarkersFileCreation failure

2020-11-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17385. - Fix Version/s: 3.3.1 Resolution: Fixed >

[jira] [Resolved] (HADOOP-17318) S3A committer to support concurrent jobs with same app attempt ID & dest dir

2020-11-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17318. - Fix Version/s: 3.3.1 Resolution: Fixed > S3A committer to support concurrent

[jira] [Resolved] (HADOOP-17313) FileSystem.get to support slow-to-instantiate FS clients

2020-11-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17313. - Fix Version/s: 3.3.1 Resolution: Fixed > FileSystem.get to support

[jira] [Resolved] (HADOOP-17325) WASB: Test failures

2020-11-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17325. - Fix Version/s: 3.3.1 Resolution: Fixed Ok, fixed in branches-3.3 and trunk.

[jira] [Resolved] (HADOOP-17343) Upgrade aws-java-sdk to 1.11.901

2020-11-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17343. - Fix Version/s: 3.3.1 Resolution: Fixed Merged to trunk and branch-3.3; PR

[jira] [Resolved] (HADOOP-17390) Skip license check on lz4 code files

2020-11-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17390. - Fix Version/s: 3.4.0 Resolution: Fixed +1, merged to trunk. Thanks! > Skip

[jira] [Resolved] (HADOOP-17388) AbstractS3ATokenIdentifier to issue date in UTC

2020-11-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17388. - Fix Version/s: 3.3.1 Resolution: Fixed +1, merged to 3.3+ >

[jira] [Created] (HADOOP-17388) AbstractS3ATokenIdentifier to issue date in UTC

2020-11-19 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17388: --- Summary: AbstractS3ATokenIdentifier to issue date in UTC Key: HADOOP-17388 URL: https://issues.apache.org/jira/browse/HADOOP-17388 Project: Hadoop Common

[jira] [Created] (HADOOP-17386) fs.s3a.buffer.dir to be under Yarn container path on yarn applications

2020-11-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17386: --- Summary: fs.s3a.buffer.dir to be under Yarn container path on yarn applications Key: HADOOP-17386 URL: https://issues.apache.org/jira/browse/HADOOP-17386

[jira] [Created] (HADOOP-17385) ITestS3ADeleteCost.testDirMarkersFileCreation failure

2020-11-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17385: --- Summary: ITestS3ADeleteCost.testDirMarkersFileCreation failure Key: HADOOP-17385 URL: https://issues.apache.org/jira/browse/HADOOP-17385 Project: Hadoop Common

[jira] [Resolved] (HADOOP-17244) HADOOP-17244. S3A directory delete tombstones dir markers prematurely.

2020-11-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17244. - Resolution: Fixed > HADOOP-17244. S3A directory delete tombstones dir markers

[jira] [Resolved] (HADOOP-17379) AbstractS3ATokenIdentifier to set issue date == now

2020-11-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17379. - Fix Version/s: 3.3.1 Resolution: Fixed > AbstractS3ATokenIdentifier to set

[jira] [Reopened] (HADOOP-17343) Upgrade aws-java-sdk to 1.11.901

2020-11-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-17343: - > Upgrade aws-java-sdk to 1.11.901 > > >

[jira] [Created] (HADOOP-17380) ITestS3AContractSeek.teardown closes FS before superclass does its cleanup

2020-11-16 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17380: --- Summary: ITestS3AContractSeek.teardown closes FS before superclass does its cleanup Key: HADOOP-17380 URL: https://issues.apache.org/jira/browse/HADOOP-17380

[jira] [Created] (HADOOP-17379) AbstractS3ATokenIdentifier to set issue date == now

2020-11-16 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17379: --- Summary: AbstractS3ATokenIdentifier to set issue date == now Key: HADOOP-17379 URL: https://issues.apache.org/jira/browse/HADOOP-17379 Project: Hadoop Common

[jira] [Resolved] (HADOOP-17134) S3AFileSystem.listLocatedStatu(file) does a LIST even with S3Guard

2020-11-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17134. - Resolution: Won't Fix > S3AFileSystem.listLocatedStatu(file) does a LIST even with

[jira] [Created] (HADOOP-17376) ITestS3AContractRename failing against stricter tests

2020-11-12 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17376: --- Summary: ITestS3AContractRename failing against stricter tests Key: HADOOP-17376 URL: https://issues.apache.org/jira/browse/HADOOP-17376 Project: Hadoop Common

[jira] [Created] (HADOOP-17372) S3A AWS Credential provider loading gets confused with isolated classloaders

2020-11-10 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17372: --- Summary: S3A AWS Credential provider loading gets confused with isolated classloaders Key: HADOOP-17372 URL: https://issues.apache.org/jira/browse/HADOOP-17372

[jira] [Resolved] (HADOOP-17066) S3A staging committer committing duplicate files

2020-11-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17066. - Resolution: Duplicate HADOOP-17318 covers the duplicate job problem everywhere in the

[jira] [Created] (HADOOP-17366) hadoop-cloud-storage transient dependencies need review

2020-11-09 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17366: --- Summary: hadoop-cloud-storage transient dependencies need review Key: HADOOP-17366 URL: https://issues.apache.org/jira/browse/HADOOP-17366 Project: Hadoop

[jira] [Created] (HADOOP-17340) TestLdapGroupsMapping failing -string mismatch in exception validation

2020-11-02 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17340: --- Summary: TestLdapGroupsMapping failing -string mismatch in exception validation Key: HADOOP-17340 URL: https://issues.apache.org/jira/browse/HADOOP-17340

[jira] [Reopened] (HADOOP-17312) S3AInputStream to be resilient to faiures in abort(); translate AWS Exceptions

2020-11-02 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-17312: - > S3AInputStream to be resilient to faiures in abort(); translate AWS Exceptions >

[jira] [Created] (HADOOP-17335) s3a listing operation will fail in async prefetch if fs closed

2020-10-28 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17335: --- Summary: s3a listing operation will fail in async prefetch if fs closed Key: HADOOP-17335 URL: https://issues.apache.org/jira/browse/HADOOP-17335 Project:

[jira] [Resolved] (HADOOP-17308) WASB : PageBlobOutputStream succeeding hflush even when underlying flush to storage failed

2020-10-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17308. - Fix Version/s: 3.3.1 Resolution: Fixed > WASB : PageBlobOutputStream succeeding

[jira] [Created] (HADOOP-17323) s3a getFileStatus("/") to skip IO

2020-10-22 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17323: --- Summary: s3a getFileStatus("/") to skip IO Key: HADOOP-17323 URL: https://issues.apache.org/jira/browse/HADOOP-17323 Project: Hadoop Common Issue

[jira] [Created] (HADOOP-17318) S3A Magic committer to make cleanup of pending uploads optional

2020-10-20 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17318: --- Summary: S3A Magic committer to make cleanup of pending uploads optional Key: HADOOP-17318 URL: https://issues.apache.org/jira/browse/HADOOP-17318 Project:

[jira] [Created] (HADOOP-17313) FileSystem.get to support slow-to-instantiate FS clients

2020-10-19 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17313: --- Summary: FileSystem.get to support slow-to-instantiate FS clients Key: HADOOP-17313 URL: https://issues.apache.org/jira/browse/HADOOP-17313 Project: Hadoop

[jira] [Created] (HADOOP-17312) S3AInputStream to be resilient to faiures in abort(); translate AWS Exceptions

2020-10-19 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17312: --- Summary: S3AInputStream to be resilient to faiures in abort(); translate AWS Exceptions Key: HADOOP-17312 URL: https://issues.apache.org/jira/browse/HADOOP-17312

[jira] [Resolved] (HADOOP-17261) s3a rename() now requires s3:deleteObjectVersion permission

2020-10-14 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17261. - Fix Version/s: 3.3.1 Resolution: Fixed > s3a rename() now requires

[jira] [Resolved] (HADOOP-17293) S3A to always probe S3 in S3A getFileStatus on non-auth paths

2020-10-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17293. - Fix Version/s: 3.3.1 Resolution: Fixed > S3A to always probe S3 in S3A

[jira] [Resolved] (HADOOP-17300) FileSystem.DirListingIterator.next() call should return NoSuchElementException

2020-10-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17300. - Fix Version/s: 3.3.1 Resolution: Fixed Fixed in HADOOP-17281, because the

[jira] [Resolved] (HADOOP-17281) Implement FileSystem.listStatusIterator() in S3AFileSystem

2020-10-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17281. - Fix Version/s: 3.3.1 Resolution: Fixed merged into 3.1+; looking forward to

[jira] [Resolved] (HADOOP-17125) Using snappy-java in SnappyCodec

2020-10-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17125. - Fix Version/s: 3.3.1 Resolution: Fixed > Using snappy-java in SnappyCodec >

[jira] [Created] (HADOOP-17293) refreshing S3Guard records after TTL-triggered-HEAD breaks some workflows

2020-10-02 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17293: --- Summary: refreshing S3Guard records after TTL-triggered-HEAD breaks some workflows Key: HADOOP-17293 URL: https://issues.apache.org/jira/browse/HADOOP-17293

[jira] [Resolved] (HADOOP-17267) Add debug-level logs in Filesystem#close

2020-09-29 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17267. - Fix Version/s: 3.3.1 Resolution: Fixed +1, merged to trunk. Thanks! > Add

[jira] [Resolved] (HADOOP-11202) SequenceFile crashes with client-side encrypted files that are shorter than FileSystem.getStatus(path)

2020-09-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-11202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-11202. - Resolution: Won't Fix > SequenceFile crashes with client-side encrypted files that are

[jira] [Resolved] (HADOOP-17023) Tune listStatus() api of s3a.

2020-09-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17023. - Fix Version/s: 3.3.1 Resolution: Fixed Available in 3.3.1+. > Tune

[jira] [Resolved] (HADOOP-17216) hadoop-aws having FileNotFoundException when accessing AWS s3 occassionally

2020-09-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17216. - Fix Version/s: 3.3.0 Resolution: Duplicate > hadoop-aws having

[jira] [Resolved] (HADOOP-15136) Typo in rename spec pseudocode

2020-09-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15136. - Fix Version/s: 3.3.1 Resolution: Fixed Fixed by david tucker. > Typo in

[jira] [Created] (HADOOP-17275) Implement Incremental globStatus() -> RemoteIterator class

2020-09-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17275: --- Summary: Implement Incremental globStatus() -> RemoteIterator class Key: HADOOP-17275 URL: https://issues.apache.org/jira/browse/HADOOP-17275 Project: Hadoop

[jira] [Created] (HADOOP-17274) MR FileInput/Output formats to aggregate IOStatistics

2020-09-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17274: --- Summary: MR FileInput/Output formats to aggregate IOStatistics Key: HADOOP-17274 URL: https://issues.apache.org/jira/browse/HADOOP-17274 Project: Hadoop Common

[jira] [Created] (HADOOP-17273) FSDataInput/Output Streams to automatically collect IOStatistics

2020-09-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17273: --- Summary: FSDataInput/Output Streams to automatically collect IOStatistics Key: HADOOP-17273 URL: https://issues.apache.org/jira/browse/HADOOP-17273 Project:

[jira] [Created] (HADOOP-17272) ABFS Streams to support IOStatistics API

2020-09-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17272: --- Summary: ABFS Streams to support IOStatistics API Key: HADOOP-17272 URL: https://issues.apache.org/jira/browse/HADOOP-17272 Project: Hadoop Common

[jira] [Created] (HADOOP-17271) S3A statistics to embrace IOStatistics

2020-09-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17271: --- Summary: S3A statistics to embrace IOStatistics Key: HADOOP-17271 URL: https://issues.apache.org/jira/browse/HADOOP-17271 Project: Hadoop Common Issue

[jira] [Created] (HADOOP-17263) transient ITestS3AFileContextStatistics failure -read buffer not filled

2020-09-16 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17263: --- Summary: transient ITestS3AFileContextStatistics failure -read buffer not filled Key: HADOOP-17263 URL: https://issues.apache.org/jira/browse/HADOOP-17263

[jira] [Created] (HADOOP-17261) s3a rename() now requires s3:deleteObjectVersion permission

2020-09-14 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17261: --- Summary: s3a rename() now requires s3:deleteObjectVersion permission Key: HADOOP-17261 URL: https://issues.apache.org/jira/browse/HADOOP-17261 Project: Hadoop

[jira] [Reopened] (HADOOP-17244) HADOOP-17244. S3A directory delete tombstones dir markers prematurely.

2020-09-11 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-17244: - seeing failures with this on cli but not IDE {code} [INFO] Running

[jira] [Resolved] (HADOOP-17244) HADOOP-17244. S3A directory delete tombstones dir markers prematurely.

2020-09-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17244. - Fix Version/s: 3.3.1 Resolution: Fixed > HADOOP-17244. S3A directory delete

[jira] [Resolved] (HADOOP-17256) DistCp -update option will be invalid when distcp files from hdfs to S3

2020-09-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17256. - Resolution: Duplicate caused by HADOOP-8143, which has now been rolled back everywhere

[jira] [Resolved] (HADOOP-17181) Handle transient stream read failures in FileSystem contract tests

2020-09-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17181. - Fix Version/s: 3.3.1 Resolution: Fixed > Handle transient stream read failures

[jira] [Resolved] (HADOOP-17189) add way for s3a to recognise buckets with "." in name and switch to path access

2020-09-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17189. - Fix Version/s: 3.4.0 Resolution: Won't Fix WONTFIXing this, as buckets with .

[jira] [Resolved] (HADOOP-17229) Test failure as failed request body counted in byte received metric - ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics

2020-09-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17229. - Fix Version/s: 3.4.0 Resolution: Fixed > Test failure as failed request body

[jira] [Resolved] (HADOOP-17158) Test timeout for ITestAbfsInputStreamStatistics#testReadAheadCounters

2020-09-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17158. - Fix Version/s: 3.4.0 Resolution: Fixed fixed in trunk; happy to cherrypick to

[jira] [Created] (HADOOP-17244) ITestS3AFileContextMainOperations#testRenameDirectoryAsNonExistentDirectory test failure on -Dauth

2020-09-04 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17244: --- Summary: ITestS3AFileContextMainOperations#testRenameDirectoryAsNonExistentDirectory test failure on -Dauth Key: HADOOP-17244 URL:

[jira] [Resolved] (HADOOP-17167) ITestS3AEncryptionWithDefaultS3Settings fails if default bucket encryption != KMS

2020-09-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17167. - Fix Version/s: 3.3.1 Resolution: Fixed in trunk / branch-3; not yet tested

[jira] [Resolved] (HADOOP-17227) improve s3guard markers command line tool

2020-09-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17227. - Fix Version/s: 3.3.1 Resolution: Fixed Fixed in branch-3.3 > improve s3guard

[jira] [Resolved] (HADOOP-17228) Backport HADOOP-13230 listing changes for preserved directory markers to 3.1.x

2020-09-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17228. - Fix Version/s: 3.1.5 Release Note: This release of Hadoop 3.1 can safely interact

[jira] [Created] (HADOOP-17242) S3A (async) ObjectListingIterator to block in hasNext() for results

2020-09-03 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17242: --- Summary: S3A (async) ObjectListingIterator to block in hasNext() for results Key: HADOOP-17242 URL: https://issues.apache.org/jira/browse/HADOOP-17242 Project:

[jira] [Resolved] (HADOOP-17241) s3a: bucket names which aren't parseable hostnames unsupported

2020-09-02 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17241. - Resolution: Duplicate I'm afraid you've just discovered the harsh truth which is the

[jira] [Resolved] (HADOOP-17194) Adding Context class for AbfsClient to pass AbfsConfigurations to limit number of parameters

2020-08-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17194. - Fix Version/s: 3.3.1 Resolution: Fixed > Adding Context class for AbfsClient to

[jira] [Created] (HADOOP-17228) Backport HADOOP-13230 listing changes for preserved directory markers to 3.1.x

2020-08-25 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17228: --- Summary: Backport HADOOP-13230 listing changes for preserved directory markers to 3.1.x Key: HADOOP-17228 URL: https://issues.apache.org/jira/browse/HADOOP-17228

[jira] [Resolved] (HADOOP-17199) Backport HADOOP-13230 list/getFileStatus changes for preserved directory markers

2020-08-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17199. - Fix Version/s: 3.2.2 Target Version/s: 3.2.2 (was: 3.2.2, 3.1.5)

[jira] [Resolved] (HADOOP-17216) hadoop-aws having FileNotFoundException when accessing AWS s3 occassionally

2020-08-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17216. - Resolution: Duplicate > hadoop-aws having FileNotFoundException when accessing AWS s3

[jira] [Created] (HADOOP-17227) improve s3guard markers -expected

2020-08-25 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17227: --- Summary: improve s3guard markers -expected Key: HADOOP-17227 URL: https://issues.apache.org/jira/browse/HADOOP-17227 Project: Hadoop Common Issue

[jira] [Created] (HADOOP-17226) Failure of ITestAssumeRole.testRestrictedCommitActions

2020-08-25 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17226: --- Summary: Failure of ITestAssumeRole.testRestrictedCommitActions Key: HADOOP-17226 URL: https://issues.apache.org/jira/browse/HADOOP-17226 Project: Hadoop

[jira] [Resolved] (HADOOP-17074) Optimise s3a Listing to be fully asynchronous.

2020-08-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17074. - Fix Version/s: 3.3.1 Resolution: Fixed > Optimise s3a Listing to be fully

[jira] [Resolved] (HADOOP-16925) MetricsConfig incorrectly loads the configuration whose value is String list in the properties file

2020-08-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16925. - Fix Version/s: 3.3.1 3.2.2 Resolution: Fixed > MetricsConfig

[jira] [Resolved] (HADOOP-17210) backport HADOOP-15691 PathCapabilities API to branch-3.2

2020-08-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17210. - Fix Version/s: 3.2.2 Resolution: Fixed > backport HADOOP-15691 PathCapabilities

[jira] [Created] (HADOOP-17210) backport HADOOP-15691 PathCapabilities API to branch-3.2

2020-08-17 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17210: --- Summary: backport HADOOP-15691 PathCapabilities API to branch-3.2 Key: HADOOP-17210 URL: https://issues.apache.org/jira/browse/HADOOP-17210 Project: Hadoop

[jira] [Resolved] (HADOOP-16493) S3AFilesystem.initiateRename() can skip check on dest.parent status if src has same parent

2020-08-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16493. - Resolution: Duplicate Ended up in HADOOP-13230. Not sure if I did that deliberately or

[jira] [Resolved] (HADOOP-17200) Renaming a file under a sibling empty directory doesn't delete dest dir's marker

2020-08-15 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17200. - Resolution: Duplicate > Renaming a file under a sibling empty directory doesn't delete

[jira] [Resolved] (HADOOP-14124) S3AFileSystem silently deletes "fake" directories when writing a file.

2020-08-15 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14124. - Fix Version/s: 3.3.1 Resolution: Fixed This feature can be disabled in Hadoop

[jira] [Resolved] (HADOOP-13230) S3A to optionally retain directory markers

2020-08-15 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13230. - Fix Version/s: 3.3.1 Hadoop Flags: Incompatible change Release Note: The S3A

[jira] [Created] (HADOOP-17207) ITestS3AConfiguration.testProxyConnection failing when s3a bucket probe disabled

2020-08-14 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17207: --- Summary: ITestS3AConfiguration.testProxyConnection failing when s3a bucket probe disabled Key: HADOOP-17207 URL: https://issues.apache.org/jira/browse/HADOOP-17207

[jira] [Resolved] (HADOOP-17192) ITestS3AHugeFilesSSECDiskBlock failing because of bucket overrides

2020-08-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17192. - Fix Version/s: 3.3.1 Resolution: Fixed > ITestS3AHugeFilesSSECDiskBlock failing

[jira] [Created] (HADOOP-17200) Renaming a file under a sibling empty directory doesn't delete dest dir's marker

2020-08-10 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17200: --- Summary: Renaming a file under a sibling empty directory doesn't delete dest dir's marker Key: HADOOP-17200 URL: https://issues.apache.org/jira/browse/HADOOP-17200

[jira] [Created] (HADOOP-17199) Backport HADOOP-13230 list/getFileStatus changes for preserved directory markers

2020-08-10 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17199: --- Summary: Backport HADOOP-13230 list/getFileStatus changes for preserved directory markers Key: HADOOP-17199 URL: https://issues.apache.org/jira/browse/HADOOP-17199

[jira] [Created] (HADOOP-17198) Support S3 Access Points

2020-08-10 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17198: --- Summary: Support S3 Access Points Key: HADOOP-17198 URL: https://issues.apache.org/jira/browse/HADOOP-17198 Project: Hadoop Common Issue Type:

[jira] [Resolved] (HADOOP-17197) Decrease size of s3a dependencies

2020-08-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17197. - Resolution: Won't Fix > Decrease size of s3a dependencies >

[jira] [Resolved] (HADOOP-13430) Optimize getFileStatus in S3A

2020-08-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13430. - Resolution: Duplicate > Optimize getFileStatus in S3A > -

[jira] [Created] (HADOOP-17189) add way for s3a to recognise buckets with "." in name and switch to path access

2020-08-06 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17189: --- Summary: add way for s3a to recognise buckets with "." in name and switch to path access Key: HADOOP-17189 URL: https://issues.apache.org/jira/browse/HADOOP-17189

[jira] [Resolved] (HADOOP-17131) Refactor S3A Listing code for better isolation

2020-08-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17131. - Fix Version/s: (was: 3.1.4) 3.3.1 Resolution: Fixed >

[jira] [Created] (HADOOP-17181) ITestS3AContractUnbuffer failure -stream.read didn't return all data

2020-08-03 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17181: --- Summary: ITestS3AContractUnbuffer failure -stream.read didn't return all data Key: HADOOP-17181 URL: https://issues.apache.org/jira/browse/HADOOP-17181

[jira] [Created] (HADOOP-17167) ITestS3AEncryptionWithDefaultS3Settings fails if default bucket encryption != KMS

2020-07-30 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17167: --- Summary: ITestS3AEncryptionWithDefaultS3Settings fails if default bucket encryption != KMS Key: HADOOP-17167 URL: https://issues.apache.org/jira/browse/HADOOP-17167

[jira] [Reopened] (HADOOP-14124) S3AFileSystem silently deletes "fake" directories when writing a file.

2020-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-14124: - > S3AFileSystem silently deletes "fake" directories when writing a file. >

[jira] [Resolved] (HADOOP-17160) ITestAbfsInputStreamStatistics#testReadAheadCounters timing out always

2020-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17160. - Resolution: Duplicate HADOOP-17158 got there first. It's interesting you always see

[jira] [Resolved] (HADOOP-13221) s3a create() doesn't check for an ancestor path being a file

2020-07-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-13221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13221. - Fix Version/s: 3.4.0 Resolution: Fixed I'm going to WONTFIX this -it's too

[jira] [Resolved] (HADOOP-17141) Add Capability To Get Text Length

2020-07-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17141. - Fix Version/s: 3.4.0 Resolution: Fixed > Add Capability To Get Text Length >

[jira] [Resolved] (HADOOP-17113) Adding ReadAhead Counters in ABFS

2020-07-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17113. - Fix Version/s: 3.4.0 Resolution: Fixed > Adding ReadAhead Counters in ABFS >

[jira] [Resolved] (HADOOP-17136) ITestS3ADirectoryPerformance.testListOperations failing

2020-07-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17136. - Resolution: Fixed > ITestS3ADirectoryPerformance.testListOperations failing >

[jira] [Resolved] (HADOOP-16930) Add com.amazonaws.auth.profile.ProfileCredentialsProvider to hadoop-aws docs

2020-07-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16930. - Fix Version/s: 3.4.0 Resolution: Fixed > Add

[jira] [Resolved] (HADOOP-17130) Configuration.getValByRegex() shouldn't update the results while fetching.

2020-07-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17130. - Fix Version/s: 3.1.5 3.3.1 3.2.2

[jira] [Created] (HADOOP-17135) ITestS3GuardOutOfBandOperations testListingDelete[auth=false] fails on unversioned bucket

2020-07-16 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17135: --- Summary: ITestS3GuardOutOfBandOperations testListingDelete[auth=false] fails on unversioned bucket Key: HADOOP-17135 URL: https://issues.apache.org/jira/browse/HADOOP-17135

[jira] [Created] (HADOOP-17134) S3AFileSystem.listLocatedStatu(file) does a LIST even with S3Guard

2020-07-16 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-17134: --- Summary: S3AFileSystem.listLocatedStatu(file) does a LIST even with S3Guard Key: HADOOP-17134 URL: https://issues.apache.org/jira/browse/HADOOP-17134 Project:

[jira] [Resolved] (HADOOP-17022) Tune S3A listFiles() api.

2020-07-15 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17022. - Fix Version/s: 3.3.1 Resolution: Fixed > Tune S3A listFiles() api. >

[jira] [Resolved] (HADOOP-16998) WASB : NativeAzureFsOutputStream#close() throwing IllegalArgumentException

2020-07-14 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16998. - Fix Version/s: 3.3.1 Resolution: Fixed Fixed in Hadoop 3.3.1 > WASB :

<    5   6   7   8   9   10   11   12   13   14   >