[jira] [Resolved] (HADOOP-15048) ITestNativeAzureFileSystemLive.testLeaseAsDistributedLock assert failure in parallel run

2017-11-17 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15048. - Resolution: Won't Fix > ITestNativeAzureFileSystemLive.testLeaseAsDistributedLock

[jira] [Created] (HADOOP-15048) ITestNativeAzureFileSystemLive.testLeaseAsDistributedLock assert failure in parallel run

2017-11-17 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15048: --- Summary: ITestNativeAzureFileSystemLive.testLeaseAsDistributedLock assert failure in parallel run Key: HADOOP-15048 URL: https://issues.apache.org/jira/browse/HADOOP-15048

[jira] [Created] (HADOOP-15050) swift:// doesn't support createNonRecursive. hence the new FS createFile(path) builder

2017-11-17 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15050: --- Summary: swift:// doesn't support createNonRecursive. hence the new FS createFile(path) builder Key: HADOOP-15050 URL: https://issues.apache.org/jira/browse/HADOOP-15050

[jira] [Created] (HADOOP-15034) S3Guard instrumentation to include cost of DynamoDB ops as metric

2017-11-13 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15034: --- Summary: S3Guard instrumentation to include cost of DynamoDB ops as metric Key: HADOOP-15034 URL: https://issues.apache.org/jira/browse/HADOOP-15034 Project:

[jira] [Created] (HADOOP-15057) s3guard bucket-info command to include default bucket encryption info

2017-11-21 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15057: --- Summary: s3guard bucket-info command to include default bucket encryption info Key: HADOOP-15057 URL: https://issues.apache.org/jira/browse/HADOOP-15057

[jira] [Resolved] (HADOOP-15055) Add s3 metrics from AWS SDK to s3a metrics tracking

2017-11-21 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15055. - Resolution: Duplicate Been on the s3a todo list for a while; assigning to you. > Add

[jira] [Resolved] (HADOOP-15051) FSDataOutputStream returned by LocalFileSystem#createNonRecursive doesn't have hflush capability

2017-11-19 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15051. - Resolution: Won't Fix > FSDataOutputStream returned by

[jira] [Created] (HADOOP-15044) Wasb getFileBlockLocations() returns too many locations.

2017-11-16 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15044: --- Summary: Wasb getFileBlockLocations() returns too many locations. Key: HADOOP-15044 URL: https://issues.apache.org/jira/browse/HADOOP-15044 Project: Hadoop

[jira] [Created] (HADOOP-15035) S3Guard to perform retry and translation of exceptions

2017-11-13 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15035: --- Summary: S3Guard to perform retry and translation of exceptions Key: HADOOP-15035 URL: https://issues.apache.org/jira/browse/HADOOP-15035 Project: Hadoop Common

[jira] [Created] (HADOOP-15015) TestConfigurationFieldsBase to use SLF4J for logging

2017-11-03 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15015: --- Summary: TestConfigurationFieldsBase to use SLF4J for logging Key: HADOOP-15015 URL: https://issues.apache.org/jira/browse/HADOOP-15015 Project: Hadoop Common

[jira] [Resolved] (HADOOP-15011) Getting file not found exception while using distcp with s3a

2017-11-02 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15011. - Resolution: Duplicate > Getting file not found exception while using distcp with s3a >

[jira] [Created] (HADOOP-15003) Merge S3A committers into trunk: Yetus patch checker

2017-11-01 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15003: --- Summary: Merge S3A committers into trunk: Yetus patch checker Key: HADOOP-15003 URL: https://issues.apache.org/jira/browse/HADOOP-15003 Project: Hadoop Common

[jira] [Created] (HADOOP-15007) Stabilize and document Configuration element

2017-11-01 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15007: --- Summary: Stabilize and document Configuration element Key: HADOOP-15007 URL: https://issues.apache.org/jira/browse/HADOOP-15007 Project: Hadoop Common

[jira] [Created] (HADOOP-15022) s3guard IT tests increase R/W capacity of the test table by 1

2017-11-07 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15022: --- Summary: s3guard IT tests increase R/W capacity of the test table by 1 Key: HADOOP-15022 URL: https://issues.apache.org/jira/browse/HADOOP-15022 Project:

[jira] [Resolved] (HADOOP-15017) is not a valid DFS filename

2017-11-06 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15017. - Resolution: Invalid > is not a valid DFS filename > > >

[jira] [Created] (HADOOP-15102) HADOOP-14831

2017-12-08 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15102: --- Summary: HADOOP-14831 Key: HADOOP-15102 URL: https://issues.apache.org/jira/browse/HADOOP-15102 Project: Hadoop Common Issue Type: Sub-task

[jira] [Created] (HADOOP-15105) add htrace context to HTTP requests as ? parameter

2017-12-08 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15105: --- Summary: add htrace context to HTTP requests as ? parameter Key: HADOOP-15105 URL: https://issues.apache.org/jira/browse/HADOOP-15105 Project: Hadoop Common

[jira] [Created] (HADOOP-15082) add AbstractContractRootDirectoryTest test for mkdir / ; wasb to implement the test

2017-12-01 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15082: --- Summary: add AbstractContractRootDirectoryTest test for mkdir / ; wasb to implement the test Key: HADOOP-15082 URL: https://issues.apache.org/jira/browse/HADOOP-15082

[jira] [Resolved] (HADOOP-14112) Über-jira adl:// Azure Data Lake Phase I: Stabilization

2017-12-04 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14112. - Resolution: Fixed Fix Version/s: 2.9.0 3.0.0 Marking as

[jira] [Resolved] (HADOOP-15086) NativeAzureFileSystem.rename is not atomic

2017-12-04 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15086. - Resolution: Won't Fix > NativeAzureFileSystem.rename is not atomic >

[jira] [Created] (HADOOP-15089) ADLS to implement AbstractContractDistCpTest

2017-12-04 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15089: --- Summary: ADLS to implement AbstractContractDistCpTest Key: HADOOP-15089 URL: https://issues.apache.org/jira/browse/HADOOP-15089 Project: Hadoop Common

[jira] [Created] (HADOOP-15090) add adl troubleshooting doc

2017-12-04 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15090: --- Summary: add adl troubleshooting doc Key: HADOOP-15090 URL: https://issues.apache.org/jira/browse/HADOOP-15090 Project: Hadoop Common Issue Type:

[jira] [Created] (HADOOP-15094) FileSystem

2017-12-06 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15094: --- Summary: FileSystem Key: HADOOP-15094 URL: https://issues.apache.org/jira/browse/HADOOP-15094 Project: Hadoop Common Issue Type: Improvement

[jira] [Created] (HADOOP-15091) S3aUtils.getEncryptionAlgorithm() always logs@Debug "Using SSE-C"

2017-12-05 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15091: --- Summary: S3aUtils.getEncryptionAlgorithm() always logs@Debug "Using SSE-C" Key: HADOOP-15091 URL: https://issues.apache.org/jira/browse/HADOOP-15091 Project:

[jira] [Resolved] (HADOOP-13967) S3ABlockOutputStream to support plugin point for different multipart strategies

2017-12-06 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13967. - Resolution: Duplicate Fix Version/s: 3.1.0 > S3ABlockOutputStream to support

[jira] [Resolved] (HADOOP-13969) S3A to support commit(path) operation, which commits all pending put commits in a path

2017-12-06 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13969. - Resolution: Duplicate Assignee: Steve Loughran Fix Version/s: 3.1.0 >

[jira] [Resolved] (HADOOP-13968) S3a FS to support "__magic" path for the special "unmaterialized" writes

2017-12-06 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13968. - Resolution: Duplicate Assignee: Steve Loughran Fix Version/s: 3.1.0 >

[jira] [Created] (HADOOP-15095) S3a committer factory to warn when default FileOutputFormat committer is created

2017-12-06 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15095: --- Summary: S3a committer factory to warn when default FileOutputFormat committer is created Key: HADOOP-15095 URL: https://issues.apache.org/jira/browse/HADOOP-15095

[jira] [Created] (HADOOP-15107) Prove the correctness of the new committers, or fix where they are not correct

2017-12-10 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15107: --- Summary: Prove the correctness of the new committers, or fix where they are not correct Key: HADOOP-15107 URL: https://issues.apache.org/jira/browse/HADOOP-15107

[jira] [Resolved] (HADOOP-15035) S3Guard to perform retry and translation of exceptions

2017-12-05 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15035. - Resolution: Duplicate > S3Guard to perform retry and translation of exceptions >

[jira] [Created] (HADOOP-14981) SnappyCodec.checkNativeCodeLoaded raises UnsatisfiedLinkError if native code is missing

2017-10-25 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-14981: --- Summary: SnappyCodec.checkNativeCodeLoaded raises UnsatisfiedLinkError if native code is missing Key: HADOOP-14981 URL: https://issues.apache.org/jira/browse/HADOOP-14981

[jira] [Created] (HADOOP-14975) S3AInputStream/OutputStream statistics aren't getting into StorageStatistics

2017-10-24 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-14975: --- Summary: S3AInputStream/OutputStream statistics aren't getting into StorageStatistics Key: HADOOP-14975 URL: https://issues.apache.org/jira/browse/HADOOP-14975

[jira] [Created] (HADOOP-14971) Merge S3A committers into trunk

2017-10-20 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-14971: --- Summary: Merge S3A committers into trunk Key: HADOOP-14971 URL: https://issues.apache.org/jira/browse/HADOOP-14971 Project: Hadoop Common Issue Type:

[jira] [Created] (HADOOP-15134) ADL problems parsing JSON responses to include error details

2017-12-20 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15134: --- Summary: ADL problems parsing JSON responses to include error details Key: HADOOP-15134 URL: https://issues.apache.org/jira/browse/HADOOP-15134 Project: Hadoop

[jira] [Resolved] (HADOOP-15138) CLONE - Executing the command 'hdfs -Dhadoop.security.credential.provider.path=file1.jceks,file2.jceks' fails if permission is denied to some files

2017-12-21 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15138. - Resolution: Duplicate Closing this as a duplicate of the original, which is the same

[jira] [Created] (HADOOP-15460) S3A FS to add "s3a:no-existence-checks" to the builder file creation option set

2018-05-14 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15460: --- Summary: S3A FS to add "s3a:no-existence-checks" to the builder file creation option set Key: HADOOP-15460 URL: https://issues.apache.org/jira/browse/HADOOP-15460

[jira] [Resolved] (HADOOP-15417) retrieveBlock hangs when the configuration file is corrupted

2018-04-27 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15417. - Resolution: Won't Fix > retrieveBlock hangs when the configuration file is corrupted >

[jira] [Created] (HADOOP-15421) Stabilise/formalise the JSON _SUCCESS format used in the S3A committers

2018-04-27 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15421: --- Summary: Stabilise/formalise the JSON _SUCCESS format used in the S3A committers Key: HADOOP-15421 URL: https://issues.apache.org/jira/browse/HADOOP-15421

[jira] [Created] (HADOOP-15422) s3guard doesnt init when the secrets are in the s3a URI

2018-04-27 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15422: --- Summary: s3guard doesnt init when the secrets are in the s3a URI Key: HADOOP-15422 URL: https://issues.apache.org/jira/browse/HADOOP-15422 Project: Hadoop

[jira] [Created] (HADOOP-15426) S3guard throttle event on delete => 400 error code => exception

2018-04-27 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15426: --- Summary: S3guard throttle event on delete => 400 error code => exception Key: HADOOP-15426 URL: https://issues.apache.org/jira/browse/HADOOP-15426 Project:

[jira] [Created] (HADOOP-15427) hadoop shell complains needlessly about "ERROR: Tools helper .. not found"

2018-04-28 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15427: --- Summary: hadoop shell complains needlessly about "ERROR: Tools helper .. not found" Key: HADOOP-15427 URL: https://issues.apache.org/jira/browse/HADOOP-15427

[jira] [Created] (HADOOP-15469) S3A directory committer commit job fails if _temporary directory created under set

2018-05-15 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15469: --- Summary: S3A directory committer commit job fails if _temporary directory created under set Key: HADOOP-15469 URL: https://issues.apache.org/jira/browse/HADOOP-15469

[jira] [Reopened] (HADOOP-15250) Split-DNS MultiHomed Server Network Cluster Network IPC Client Bind Addr Wrong

2018-05-15 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-15250: - > Split-DNS MultiHomed Server Network Cluster Network IPC Client Bind Addr Wrong >

[jira] [Created] (HADOOP-15470) S3A staging committers to not log FNFEs on job abort listings

2018-05-15 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15470: --- Summary: S3A staging committers to not log FNFEs on job abort listings Key: HADOOP-15470 URL: https://issues.apache.org/jira/browse/HADOOP-15470 Project:

[jira] [Created] (HADOOP-15485) reduce/tune read failure fault injection on inconsistent client

2018-05-21 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15485: --- Summary: reduce/tune read failure fault injection on inconsistent client Key: HADOOP-15485 URL: https://issues.apache.org/jira/browse/HADOOP-15485 Project:

[jira] [Created] (HADOOP-15489) S3Guard to self update on directory listings of S3

2018-05-22 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15489: --- Summary: S3Guard to self update on directory listings of S3 Key: HADOOP-15489 URL: https://issues.apache.org/jira/browse/HADOOP-15489 Project: Hadoop Common

[jira] [Created] (HADOOP-15492) increase performance of s3guard import command

2018-05-24 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15492: --- Summary: increase performance of s3guard import command Key: HADOOP-15492 URL: https://issues.apache.org/jira/browse/HADOOP-15492 Project: Hadoop Common

[jira] [Created] (HADOOP-15479) s3guard bucket-info command to add a verify-property =

2018-05-18 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15479: --- Summary: s3guard bucket-info command to add a verify-property = Key: HADOOP-15479 URL: https://issues.apache.org/jira/browse/HADOOP-15479 Project: Hadoop

[jira] [Created] (HADOOP-15540) ABFS: Commit of core codebase

2018-06-14 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15540: --- Summary: ABFS: Commit of core codebase Key: HADOOP-15540 URL: https://issues.apache.org/jira/browse/HADOOP-15540 Project: Hadoop Common Issue Type:

[jira] [Resolved] (HADOOP-15542) S3AFileSystem - FileAlreadyExistsException when prefix is a file and part of a directory tree

2018-06-15 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15542. - Resolution: Won't Fix > S3AFileSystem - FileAlreadyExistsException when prefix is a

[jira] [Created] (HADOOP-15544) ABFS: validate packing, transient classpath, hadoop fs CLI

2018-06-15 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15544: --- Summary: ABFS: validate packing, transient classpath, hadoop fs CLI Key: HADOOP-15544 URL: https://issues.apache.org/jira/browse/HADOOP-15544 Project: Hadoop

[jira] [Created] (HADOOP-15545) ABFS initialize() throws string out of bounds exception of the URI isn't fully qualified

2018-06-15 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15545: --- Summary: ABFS initialize() throws string out of bounds exception of the URI isn't fully qualified Key: HADOOP-15545 URL: https://issues.apache.org/jira/browse/HADOOP-15545

[jira] [Created] (HADOOP-15546) ABFS: tune imports & javadocs

2018-06-15 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15546: --- Summary: ABFS: tune imports & javadocs Key: HADOOP-15546 URL: https://issues.apache.org/jira/browse/HADOOP-15546 Project: Hadoop Common Issue Type:

[jira] [Created] (HADOOP-15503) strip s3.amazonaws.com off hostnames before making s3a calls

2018-05-30 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15503: --- Summary: strip s3.amazonaws.com off hostnames before making s3a calls Key: HADOOP-15503 URL: https://issues.apache.org/jira/browse/HADOOP-15503 Project: Hadoop

[jira] [Created] (HADOOP-15509) Release Hadoop 2.7.7

2018-06-02 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15509: --- Summary: Release Hadoop 2.7.7 Key: HADOOP-15509 URL: https://issues.apache.org/jira/browse/HADOOP-15509 Project: Hadoop Common Issue Type: Task

[jira] [Created] (HADOOP-15512) clean up Shell from JDK7 workarounds

2018-06-04 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15512: --- Summary: clean up Shell from JDK7 workarounds Key: HADOOP-15512 URL: https://issues.apache.org/jira/browse/HADOOP-15512 Project: Hadoop Common Issue

[jira] [Created] (HADOOP-15575) Add a way for an FS instance to say "really, no trash interval at all"

2018-07-02 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15575: --- Summary: Add a way for an FS instance to say "really, no trash interval at all" Key: HADOOP-15575 URL: https://issues.apache.org/jira/browse/HADOOP-15575

[jira] [Created] (HADOOP-15576) S3A Multipart Uploader API implementation to work with S3Guard

2018-07-02 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15576: --- Summary: S3A Multipart Uploader API implementation to work with S3Guard Key: HADOOP-15576 URL: https://issues.apache.org/jira/browse/HADOOP-15576 Project:

[jira] [Created] (HADOOP-15563) s3guard init and set-capacity to support DDB autoscaling

2018-06-26 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15563: --- Summary: s3guard init and set-capacity to support DDB autoscaling Key: HADOOP-15563 URL: https://issues.apache.org/jira/browse/HADOOP-15563 Project: Hadoop

[jira] [Resolved] (HADOOP-13811) s3a: getFileStatus fails with com.amazonaws.AmazonClientException: Failed to sanitize XML document destined for handler class

2018-06-27 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13811. - Resolution: Cannot Reproduce No Vinod, this is nothing to do with HADOOP-13786 except

[jira] [Reopened] (HADOOP-13811) s3a: getFileStatus fails with com.amazonaws.AmazonClientException: Failed to sanitize XML document destined for handler class

2018-06-27 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-13811: - > s3a: getFileStatus fails with com.amazonaws.AmazonClientException: Failed to > sanitize

[jira] [Resolved] (HADOOP-13844) Move swift duration stats tracking to hadoop common utils

2018-06-25 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13844. - Resolution: Won't Fix > Move swift duration stats tracking to hadoop common utils >

[jira] [Resolved] (HADOOP-14614) Hive doesn't let s3a patch the credential provider path

2018-07-02 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14614. - Resolution: Won't Fix we can't fix this: put fully qualified account/secrets in your

[jira] [Created] (HADOOP-15439) KDiag to add cli options for proxies and taking to HDFS for listing & DT retrieval

2018-05-02 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15439: --- Summary: KDiag to add cli options for proxies and taking to HDFS for listing & DT retrieval Key: HADOOP-15439 URL: https://issues.apache.org/jira/browse/HADOOP-15439

[jira] [Created] (HADOOP-15416) s3guard diff assert failure

2018-04-26 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15416: --- Summary: s3guard diff assert failure Key: HADOOP-15416 URL: https://issues.apache.org/jira/browse/HADOOP-15416 Project: Hadoop Common Issue Type:

[jira] [Created] (HADOOP-15430) hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard

2018-04-30 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15430: --- Summary: hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard Key: HADOOP-15430 URL: https://issues.apache.org/jira/browse/HADOOP-15430 Project:

[jira] [Created] (HADOOP-15428) s3guard bucket-info -unguarded will guard bucket if FS is set to do this automatically

2018-04-28 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15428: --- Summary: s3guard bucket-info -unguarded will guard bucket if FS is set to do this automatically Key: HADOOP-15428 URL: https://issues.apache.org/jira/browse/HADOOP-15428

[jira] [Resolved] (HADOOP-15577) Update distcp to use zero-rename s3 committers

2018-07-03 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15577. - Resolution: Duplicate We don't need the 0-rename stuff, because distcp, except in the

[jira] [Resolved] (HADOOP-15163) Fix S3ACommitter documentation

2018-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15163. - Resolution: Fixed Fix Version/s: 3.1.0 LGTM, +1, committed. thanks > Fix

[jira] [Created] (HADOOP-15176) Enhance IAM assumed role support in S3A client

2018-01-17 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15176: --- Summary: Enhance IAM assumed role support in S3A client Key: HADOOP-15176 URL: https://issues.apache.org/jira/browse/HADOOP-15176 Project: Hadoop Common

[jira] [Created] (HADOOP-15183) S3Guard store becomes inconsistent after partial failure of rename

2018-01-19 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15183: --- Summary: S3Guard store becomes inconsistent after partial failure of rename Key: HADOOP-15183 URL: https://issues.apache.org/jira/browse/HADOOP-15183 Project:

[jira] [Created] (HADOOP-15191) Add Private/Unstable BulkDelete operations to FS/S3A for DistCP

2018-01-24 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15191: --- Summary: Add Private/Unstable BulkDelete operations to FS/S3A for DistCP Key: HADOOP-15191 URL: https://issues.apache.org/jira/browse/HADOOP-15191 Project:

[jira] [Resolved] (HADOOP-15192) S3A listStatus excessively slow

2018-01-25 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15192. - Resolution: Duplicate > S3A listStatus excessively slow >

[jira] [Created] (HADOOP-15193) add bulk delete call to metastore API & DDB impl

2018-01-25 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15193: --- Summary: add bulk delete call to metastore API & DDB impl Key: HADOOP-15193 URL: https://issues.apache.org/jira/browse/HADOOP-15193 Project: Hadoop Common

[jira] [Resolved] (HADOOP-13059) S3a over-reacts to potentially transient network problems in its init() logic

2018-01-12 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13059. - Resolution: Duplicate Fix Version/s: 3.1.0 This comes with HADOOP-14303 > S3a

[jira] [Resolved] (HADOOP-14353) add an SSE-KMS scale test to see if you can overload the keystore in random IO

2018-01-31 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14353. - Resolution: Won't Fix not seen this happening in the field, doubt you can do it for a

[jira] [Created] (HADOOP-15201) Automatically determine region & hence S3 endpoint of buckets

2018-01-31 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15201: --- Summary: Automatically determine region & hence S3 endpoint of buckets Key: HADOOP-15201 URL: https://issues.apache.org/jira/browse/HADOOP-15201 Project:

[jira] [Created] (HADOOP-15209) PoC: DistCp to eliminate needless deletion of files under deleted directories

2018-02-05 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15209: --- Summary: PoC: DistCp to eliminate needless deletion of files under deleted directories Key: HADOOP-15209 URL: https://issues.apache.org/jira/browse/HADOOP-15209

[jira] [Created] (HADOOP-15216) S3AInputStream to handle reconnect on read() failure better

2018-02-08 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15216: --- Summary: S3AInputStream to handle reconnect on read() failure better Key: HADOOP-15216 URL: https://issues.apache.org/jira/browse/HADOOP-15216 Project: Hadoop

[jira] [Created] (HADOOP-15215) s3guard set-capacity command to fail on read/write of 0

2018-02-07 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15215: --- Summary: s3guard set-capacity command to fail on read/write of 0 Key: HADOOP-15215 URL: https://issues.apache.org/jira/browse/HADOOP-15215 Project: Hadoop

[jira] [Created] (HADOOP-15210) Handle FNFE from S3Guard.getMetadataStore() in S3A initialize()

2018-02-05 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15210: --- Summary: Handle FNFE from S3Guard.getMetadataStore() in S3A initialize() Key: HADOOP-15210 URL: https://issues.apache.org/jira/browse/HADOOP-15210 Project:

[jira] [Created] (HADOOP-15220) Über-jira: S3a phase V: Hadoop 3.2 features

2018-02-12 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15220: --- Summary: Über-jira: S3a phase V: Hadoop 3.2 features Key: HADOOP-15220 URL: https://issues.apache.org/jira/browse/HADOOP-15220 Project: Hadoop Common

[jira] [Resolved] (HADOOP-14736) S3AInputStream to implement an efficient skip() call through seeking

2018-02-12 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14736. - Resolution: Duplicate I should fix HADOOP-14606 instead of filing the same bug whenever

[jira] [Resolved] (HADOOP-13892) use s3 tags/headers to record permissions on objects, so preserving them through distcp round trips

2018-02-12 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13892. - Resolution: Won't Fix > use s3 tags/headers to record permissions on objects, so

[jira] [Resolved] (HADOOP-13371) S3A globber to use bulk listObject call over recursive directory scan

2018-02-12 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13371. - Resolution: Won't Fix > S3A globber to use bulk listObject call over recursive

[jira] [Resolved] (HADOOP-13648) s3a home directory to be "/"

2018-02-12 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13648. - Resolution: Won't Fix too late to fix this, I suspect > s3a home directory to be "/" >

[jira] [Resolved] (HADOOP-13974) S3Guard CLI to support list/purge of pending multipart commits

2018-02-12 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13974. - Resolution: Fixed > S3Guard CLI to support list/purge of pending multipart commits >

[jira] [Resolved] (HADOOP-14975) S3AInputStream/OutputStream statistics aren't getting into StorageStatistics

2018-02-12 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14975. - Resolution: Duplicate Duplicate of HADOOP-15161; that is fixed so closing this one >

[jira] [Created] (HADOOP-15224) builld up md5 checksum as blocks are built in S3ABlockOutputStream; validate upload

2018-02-13 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15224: --- Summary: builld up md5 checksum as blocks are built in S3ABlockOutputStream; validate upload Key: HADOOP-15224 URL: https://issues.apache.org/jira/browse/HADOOP-15224

[jira] [Resolved] (HADOOP-13271) Intermittent failure of TestS3AContractRootDir.testListEmptyRootDirectory

2018-02-13 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13271. - Resolution: Cannot Reproduce Fix Version/s: 3.1.0 > Intermittent failure of

[jira] [Created] (HADOOP-15226) Über-JIRA: S3Guard Phase III: Hadoop 3.2 features

2018-02-13 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15226: --- Summary: Über-JIRA: S3Guard Phase III: Hadoop 3.2 features Key: HADOOP-15226 URL: https://issues.apache.org/jira/browse/HADOOP-15226 Project: Hadoop Common

[jira] [Resolved] (HADOOP-14531) Improve S3A error handling & reporting

2018-02-13 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14531. - Resolution: Fixed Fix Version/s: 3.1.0 > Improve S3A error handling & reporting

[jira] [Resolved] (HADOOP-13973) S3A GET/HEAD requests failing: java.lang.IllegalStateException: Connection is not open/Connection pool shut down

2018-02-13 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13973. - Resolution: Cannot Reproduce Fix Version/s: 3.1.0 Haven't seen this ourselves,

[jira] [Resolved] (HADOOP-13713) ITestS3AContractRootDir.testRmEmptyRootDirNonRecursive failing intermittently

2018-02-13 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13713. - Resolution: Cannot Reproduce Target Version/s: 3.1.0 >

[jira] [Resolved] (HADOOP-13308) S3A delete and rename may fail to preserve parent directory.

2018-02-13 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13308. - Resolution: Won't Fix > S3A delete and rename may fail to preserve parent directory. >

[jira] [Resolved] (HADOOP-14620) S3A authentication failure for regions other than us-east-1

2018-02-13 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14620. - Resolution: Works for Me Fix Version/s: 3.1.0 > S3A authentication failure for

[jira] [Created] (HADOOP-15227) add mapreduce.outputcommitter.factory.scheme.s3a to core-default

2018-02-13 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15227: --- Summary: add mapreduce.outputcommitter.factory.scheme.s3a to core-default Key: HADOOP-15227 URL: https://issues.apache.org/jira/browse/HADOOP-15227 Project:

[jira] [Created] (HADOOP-15228) S3A Retry policy to retry on NoResponseException

2018-02-13 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15228: --- Summary: S3A Retry policy to retry on NoResponseException Key: HADOOP-15228 URL: https://issues.apache.org/jira/browse/HADOOP-15228 Project: Hadoop Common

[jira] [Resolved] (HADOOP-14332) Document S3A SSE

2018-02-13 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14332. - Resolution: Duplicate Assignee: Steve Loughran > Document S3A SSE >

[jira] [Resolved] (HADOOP-14325) Stabilise S3A Server Side Encryption

2018-02-13 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14325. - Resolution: Fixed Fix Version/s: 3.1.0 > Stabilise S3A Server Side Encryption >

<    4   5   6   7   8   9   10   11   12   13   >