[jira] [Updated] (HADOOP-18915) S3A client builder to use getTImeDuration() for HTTP timeouts

2023-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18915: Summary: S3A client builder to use getTImeDuration() for HTTP timeouts (was: HTTP

[jira] [Commented] (HADOOP-18915) HTTP timeouts are not set correctly

2023-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779616#comment-17779616 ] Steve Loughran commented on HADOOP-18915: - so HADOOP-18889 does switch from seconds to millis,

[jira] [Created] (HADOOP-18951) S3A third party: document "Certificate doesn't match"

2023-10-25 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18951: --- Summary: S3A third party: document "Certificate doesn't match" Key: HADOOP-18951 URL: https://issues.apache.org/jira/browse/HADOOP-18951 Project: Hadoop Common

[jira] [Resolved] (HADOOP-18948) S3A. Add option fs.s3a.directory.operations.purge.uploads to purge on rename/delete

2023-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18948. - Fix Version/s: 3.4.0 Release Note: S3A directory delete and rename will

[jira] [Assigned] (HADOOP-18933) upgrade netty to 4.1.100 due to CVE

2023-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18933: --- Assignee: PJ Fanning > upgrade netty to 4.1.100 due to CVE >

[jira] [Resolved] (HADOOP-18933) upgrade netty to 4.1.100 due to CVE

2023-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18933. - Resolution: Fixed > upgrade netty to 4.1.100 due to CVE >

[jira] [Updated] (HADOOP-18933) upgrade netty to 4.1.100 due to CVE

2023-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18933: Component/s: build Fix Version/s: 3.4.0 Description: follow up to

[jira] [Resolved] (HADOOP-18932) Upgrade AWS v2 SDK to 2.20.160 and v1 to 1.12.565

2023-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18932. - Fix Version/s: 3.4.0 Assignee: Steve Loughran Resolution: Fixed >

[jira] [Updated] (HADOOP-18932) Upgrade AWS v2 SDK to 2.20.160 and v1 to 1.12.565

2023-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18932: Summary: Upgrade AWS v2 SDK to 2.20.160 and v1 to 1.12.565 (was: upgrade AWS v1 and v2

[jira] [Updated] (HADOOP-18948) S3A. Add option fs.s3a.directory.operations.purge.uploads to purge on rename/delete

2023-10-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18948: Summary: S3A. Add option fs.s3a.directory.operations.purge.uploads to purge on

[jira] [Resolved] (HADOOP-18949) upgrade maven dependency plugin due to security issue

2023-10-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18949. - Fix Version/s: 3.4.0 Assignee: PJ Fanning Resolution: Fixed > upgrade

[jira] [Commented] (HADOOP-18948) S3A. Add option fs.s3a.directory.operations.purge.uploads to do that on rename/delete

2023-10-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17778824#comment-17778824 ] Steve Loughran commented on HADOOP-18948: - # surprised we didn't have this earlier' better than

[jira] [Created] (HADOOP-18948) S3A. Add option fs.s3a.directory.operations.purge.uploads to do that on rename/delete

2023-10-23 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18948: --- Summary: S3A. Add option fs.s3a.directory.operations.purge.uploads to do that on rename/delete Key: HADOOP-18948 URL: https://issues.apache.org/jira/browse/HADOOP-18948

[jira] [Resolved] (HADOOP-18939) NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry()

2023-10-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18939. - Fix Version/s: 3.4.0 Assignee: Steve Loughran Resolution: Fixed > NPE

[jira] [Resolved] (HADOOP-18945) S3A: IAMInstanceCredentialsProvider failing: Failed to load credentials from IMDS

2023-10-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18945. - Fix Version/s: 3.4.0 Resolution: Fixed > S3A: IAMInstanceCredentialsProvider

[jira] [Commented] (HADOOP-18862) [JDK17] MiniYarnClusters don't launch in hadoop-aws integration tests

2023-10-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1651#comment-1651 ] Steve Loughran commented on HADOOP-18862: - should be fixed now. i'll need to retest > [JDK17]

[jira] [Resolved] (HADOOP-18946) S3A: testMultiObjectExceptionFilledIn() assertion error

2023-10-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18946. - Fix Version/s: 3.4.0 Resolution: Fixed > S3A:

[jira] [Assigned] (HADOOP-18946) S3A: testMultiObjectExceptionFilledIn() assertion error

2023-10-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18946: --- Assignee: Steve Loughran > S3A: testMultiObjectExceptionFilledIn() assertion

[jira] [Created] (HADOOP-18946) S3A: testMultiObjectExceptionFilledIn() assertion error

2023-10-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18946: --- Summary: S3A: testMultiObjectExceptionFilledIn() assertion error Key: HADOOP-18946 URL: https://issues.apache.org/jira/browse/HADOOP-18946 Project: Hadoop

[jira] [Commented] (HADOOP-18939) NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry()

2023-10-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776741#comment-17776741 ] Steve Loughran commented on HADOOP-18939: - aah, added a new failure from an assert that is

[jira] [Commented] (HADOOP-18945) S3A: IAMInstanceCredentialsProvider failing: Failed to load credentials from IMDS

2023-10-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776607#comment-17776607 ] Steve Loughran commented on HADOOP-18945: - looking for that error string "Failed to load

[jira] [Updated] (HADOOP-18945) S3A: IAMInstanceCredentialsProvider failing: Failed to load credentials from IMDS

2023-10-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18945: Description: Failures in impala test VMs using iAM for auth {code} Failed to open file

[jira] [Updated] (HADOOP-18945) S3A: IAMInstanceCredentialsProvider failing: Failed to load credentials from IMDS

2023-10-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18945: Summary: S3A: IAMInstanceCredentialsProvider failing: Failed to load credentials from

[jira] [Created] (HADOOP-18945) S3A: IAMInstanceCredentialsProvider failing: NoAwsCredentialsException: +: Failed to load credentials from IMDS.{noformat}

2023-10-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18945: --- Summary: S3A: IAMInstanceCredentialsProvider failing: NoAwsCredentialsException: +: Failed to load credentials from IMDS.{noformat} Key: HADOOP-18945 URL:

[jira] [Commented] (HADOOP-18944) s3a openfile is using readahead rather than async drain threshold to calculate drain threshold

2023-10-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776591#comment-17776591 ] Steve Loughran commented on HADOOP-18944: - one line fix and I don't even think we need a test

[jira] [Updated] (HADOOP-18944) s3a openfile is using readahead rather than async drain threshold to calculate drain threshold

2023-10-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18944: Description: we are using the wrong default value for the async drain threshold in

[jira] [Created] (HADOOP-18944) s3a openfile is using readahead rather than async drain threshold to calculate drain threshold

2023-10-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18944: --- Summary: s3a openfile is using readahead rather than async drain threshold to calculate drain threshold Key: HADOOP-18944 URL:

[jira] [Commented] (HADOOP-18938) S3A region logic to handle vpce and non standard endpoints

2023-10-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776243#comment-17776243 ] Steve Loughran commented on HADOOP-18938: - fail fast may not be the right strategy as the

[jira] [Resolved] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

2023-10-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18908. - Fix Version/s: 3.4.0 Resolution: Fixed > Improve s3a region handling, including

[jira] [Updated] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

2023-10-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18908: Description: S3A region logic improved for better inference and to be compatible with

[jira] [Created] (HADOOP-18940) ABFS: Remove commons IOUtils.close() from AbfsOutputStream

2023-10-17 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18940: --- Summary: ABFS: Remove commons IOUtils.close() from AbfsOutputStream Key: HADOOP-18940 URL: https://issues.apache.org/jira/browse/HADOOP-18940 Project: Hadoop

[jira] [Commented] (HADOOP-18938) S3A region logic to handle vpce and non standard endpoints

2023-10-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775842#comment-17775842 ] Steve Loughran commented on HADOOP-18938: - followup to HADOOP-18908 > S3A region logic to

[jira] [Updated] (HADOOP-18938) S3A region logic to handle vpce and non standard endpoints

2023-10-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18938: Summary: S3A region logic to handle vpce and non standard endpoints (was: Handle non

[jira] [Commented] (HADOOP-18939) NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry()

2023-10-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775773#comment-17775773 ] Steve Loughran commented on HADOOP-18939: - i can replicate this with MultiObjectDeleteException

[jira] [Commented] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from

2023-10-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775703#comment-17775703 ] Steve Loughran commented on HADOOP-18931: - this is service loader only, so for those where we

[jira] [Created] (HADOOP-18939) NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry()

2023-10-16 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18939: --- Summary: NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry() Key: HADOOP-18939 URL: https://issues.apache.org/jira/browse/HADOOP-18939 Project: Hadoop

[jira] [Updated] (HADOOP-18487) protobuf-2.5.0 dependencies => provided

2023-10-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18487: Fix Version/s: 3.4.0 > protobuf-2.5.0 dependencies => provided >

[jira] [Assigned] (HADOOP-18487) Make protobuf 2.5 an optional runtime dependency.

2023-10-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18487: --- Assignee: Steve Loughran > Make protobuf 2.5 an optional runtime dependency. >

[jira] [Updated] (HADOOP-18487) Make protobuf 2.5 an optional runtime dependency.

2023-10-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18487: Summary: Make protobuf 2.5 an optional runtime dependency. (was: protobuf-2.5.0

[jira] [Commented] (HADOOP-18487) protobuf-2.5.0 dependencies => provided

2023-10-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774902#comment-17774902 ] Steve Loughran commented on HADOOP-18487: - Part 1 PR of this is in trunk and up for 3.3: it is

[jira] [Resolved] (HADOOP-18890) remove okhttp usage

2023-10-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18890. - Fix Version/s: 3.3.9 Resolution: Fixed merged and backported. [~mukund-thakur]

[jira] [Resolved] (HADOOP-18857) AWS v2 SDK: fail meaningfully if legacy v2 signing requested

2023-10-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18857. - Fix Version/s: 3.4.0 Resolution: Fixed > AWS v2 SDK: fail meaningfully if

[jira] [Resolved] (HADOOP-18927) S3ARetryHandler to treat SocketExceptions as connectivity failures

2023-10-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18927. - Fix Version/s: 3.4.0 Resolution: Fixed > S3ARetryHandler to treat

[jira] [Created] (HADOOP-18935) coalesce AWS S3 client proxy settings

2023-10-12 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18935: --- Summary: coalesce AWS S3 client proxy settings Key: HADOOP-18935 URL: https://issues.apache.org/jira/browse/HADOOP-18935 Project: Hadoop Common Issue

[jira] [Commented] (HADOOP-18915) HTTP timeouts are not set correctly

2023-10-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774416#comment-17774416 ] Steve Loughran commented on HADOOP-18915: - actually there's more than the patch One of the

[jira] [Assigned] (HADOOP-18927) S3ARetryHandler to treat SocketExceptions as connectivity failures

2023-10-11 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18927: --- Assignee: Steve Loughran > S3ARetryHandler to treat SocketExceptions as

[jira] [Created] (HADOOP-18932) upgrade AWS v1 and v2 SDKs

2023-10-11 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18932: --- Summary: upgrade AWS v1 and v2 SDKs Key: HADOOP-18932 URL: https://issues.apache.org/jira/browse/HADOOP-18932 Project: Hadoop Common Issue Type:

[jira] [Created] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from

2023-10-11 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18931: --- Summary: FileSystem.getFileSystemClass() to log at debug the jar the .class came from Key: HADOOP-18931 URL: https://issues.apache.org/jira/browse/HADOOP-18931

[jira] [Commented] (HADOOP-18930) make fs.s3a.create.performance an option you can set for the entire bucket

2023-10-11 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774038#comment-17774038 ] Steve Loughran commented on HADOOP-18930: - thinking of making the path capability probe show

[jira] [Created] (HADOOP-18930) make fs.s3a.create.performance an option you can set for the entire bucket

2023-10-10 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18930: --- Summary: make fs.s3a.create.performance an option you can set for the entire bucket Key: HADOOP-18930 URL: https://issues.apache.org/jira/browse/HADOOP-18930

[jira] [Resolved] (HADOOP-18747) AWS SDK V2 - sigv2 support

2023-10-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18747. - Hadoop Flags: Incompatible change Release Note: AWS v2 signing is no longer

[jira] [Assigned] (HADOOP-18915) HTTP timeouts are not set correctly

2023-10-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18915: --- Assignee: Steve Loughran > HTTP timeouts are not set correctly >

[jira] [Commented] (HADOOP-18915) HTTP timeouts are not set correctly

2023-10-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773736#comment-17773736 ] Steve Loughran commented on HADOOP-18915: - let me fix in HADOOP-18889; it may have been part of

[jira] [Commented] (HADOOP-18927) S3ARetryHandler to treat SocketExceptions as connectivity failures

2023-10-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773628#comment-17773628 ] Steve Loughran commented on HADOOP-18927: - * treat NoRouteToHostException and BindException as

[jira] [Commented] (HADOOP-18927) S3ARetryHandler to treat SocketExceptions as connectivity failures

2023-10-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773622#comment-17773622 ] Steve Loughran commented on HADOOP-18927: - full v1 stack. I know this is v1, but v2 is where we

[jira] [Created] (HADOOP-18927) S3ARetryHandler to treat SocketExceptions as connectivity failures

2023-10-10 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18927: --- Summary: S3ARetryHandler to treat SocketExceptions as connectivity failures Key: HADOOP-18927 URL: https://issues.apache.org/jira/browse/HADOOP-18927 Project:

[jira] [Updated] (HADOOP-18869) ABFS: Fixing Behavior of a File System APIs on root path

2023-10-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18869: Fix Version/s: 3.4.0 > ABFS: Fixing Behavior of a File System APIs on root path >

[jira] [Resolved] (HADOOP-18861) ABFS: Fix failing CPK tests on trunk

2023-10-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18861. - Fix Version/s: 3.4.0 Resolution: Fixed > ABFS: Fix failing CPK tests on trunk >

[jira] [Commented] (HADOOP-18872) ABFS: Misreporting Retry Count for Sub-sequential and Parallel Operations

2023-10-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773406#comment-17773406 ] Steve Loughran commented on HADOOP-18872: - [~asrani_anmol] can you leave that "fix version"

[jira] [Updated] (HADOOP-18872) ABFS: Misreporting Retry Count for Sub-sequential and Parallel Operations

2023-10-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18872: Fix Version/s: (was: 3.3.6) > ABFS: Misreporting Retry Count for Sub-sequential and

[jira] [Resolved] (HADOOP-18876) ABFS: Change default from disk to bytebuffer for fs.azure.data.blocks.buffer

2023-10-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18876. - Fix Version/s: 3.4.0 (was: 3.3.6) Release Note: The

[jira] [Updated] (HADOOP-18925) S3A: add option "fs.s3a.copy.from.local.enabled" to enable/disable CopyFromLocalOperation

2023-10-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18925: Summary: S3A: add option "fs.s3a.copy.from.local.enabled" to enable/disable

[jira] [Updated] (HADOOP-18925) s3a: add option "fs.s3a.copy.from.local.enabled" to enable/disable CopyFromLocalOperation

2023-10-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18925: Summary: s3a: add option "fs.s3a.copy.from.local.enabled" to enable/disable

[jira] [Created] (HADOOP-18925) s3a: add option to disable CopyFromLocalOperation

2023-10-09 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18925: --- Summary: s3a: add option to disable CopyFromLocalOperation Key: HADOOP-18925 URL: https://issues.apache.org/jira/browse/HADOOP-18925 Project: Hadoop Common

[jira] [Commented] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store

2023-10-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772581#comment-17772581 ] Steve Loughran commented on HADOOP-18889: - update: i don't see connection refusal failing fast

[jira] [Resolved] (HADOOP-18921) upgrade avro in hadoop-thirdparty to 1.11.3

2023-10-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18921. - Assignee: PJ Fanning Resolution: Fixed > upgrade avro in hadoop-thirdparty to

[jira] [Updated] (HADOOP-18921) upgrade avro in hadoop-thirdparty to 1.11.3

2023-10-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18921: Affects Version/s: thirdparty-1.2.0 > upgrade avro in hadoop-thirdparty to 1.11.3 >

[jira] [Updated] (HADOOP-18921) upgrade avro in hadoop-thirdparty to 1.11.3

2023-10-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18921: Fix Version/s: thirdparty-1.2.0 > upgrade avro in hadoop-thirdparty to 1.11.3 >

[jira] [Updated] (HADOOP-18921) upgrade avro in hadoop-thirdparty to 1.11.3

2023-10-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18921: Component/s: build > upgrade avro in hadoop-thirdparty to 1.11.3 >

[jira] [Commented] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store

2023-10-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772240#comment-17772240 ] Steve Loughran commented on HADOOP-18889: - actually, the stack is {code}

[jira] [Commented] (HADOOP-18877) AWS SDK V2 - Move to S3 Java async client

2023-10-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771859#comment-17771859 ] Steve Loughran commented on HADOOP-18877: - +we can actually add a minimal stub implementation

[jira] [Commented] (HADOOP-18830) Cut S3 Select

2023-10-02 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771164#comment-17771164 ] Steve Loughran commented on HADOOP-18830: - going to cut this. it doesn't get used as predicate

[jira] [Updated] (HADOOP-18830) Cut S3 Select

2023-10-02 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18830: Summary: Cut S3 Select (was: S3 Select: deprecate vs cut) > Cut S3 Select >

[jira] [Commented] (HADOOP-18857) AWS v2 SDK: fail meaningfully if legacy v2 signing requested

2023-09-29 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770526#comment-17770526 ] Steve Loughran commented on HADOOP-18857: - the test is added. now InstantiationIOException is

[jira] [Assigned] (HADOOP-18857) AWS v2 SDK: fail meaningfully if legacy v2 signing requested

2023-09-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18857: --- Assignee: Steve Loughran > AWS v2 SDK: fail meaningfully if legacy v2 signing

[jira] [Resolved] (HADOOP-18758) build/release a shaded version of the AWS SDK

2023-09-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18758. - Resolution: Won't Fix let's not try this, it'll only add extra work and introduce

[jira] [Commented] (HADOOP-18857) AWS v2 SDK: fail meaningfully if legacy v2 signing requested

2023-09-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769969#comment-17769969 ] Steve Loughran commented on HADOOP-18857: - fixed in HADOOP-18889 though i need a new test >

[jira] [Resolved] (HADOOP-18871) S3ARetryPolicy to use sdk exception retryable() if it is valid

2023-09-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18871. - Resolution: Won't Fix no, wontfix. sdk thinks things like UnknownHostException are

[jira] [Commented] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store

2023-09-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768039#comment-17768039 ] Steve Loughran commented on HADOOP-18889: - npe will at least be fixed {code} ^C^C2023-09-22

[jira] [Commented] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store

2023-09-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768038#comment-17768038 ] Steve Loughran commented on HADOOP-18889: - +if the hostname is valid but no service there, sdk

[jira] [Commented] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store

2023-09-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768032#comment-17768032 ] Steve Loughran commented on HADOOP-18889: - * pulling in HADOOP-18857 as its related * if I ask

[jira] [Commented] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store

2023-09-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767666#comment-17767666 ] Steve Loughran commented on HADOOP-18889: - [~ahmarsu] thanks; will look at tomorrow. one

[jira] [Commented] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store

2023-09-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767563#comment-17767563 ] Steve Loughran commented on HADOOP-18889: - region checking of aa 3rd party bucket with no s3

[jira] [Commented] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store

2023-09-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767265#comment-17767265 ] Steve Loughran commented on HADOOP-18889: - +500 errors are retried in the aws sdk with the

[jira] [Assigned] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store

2023-09-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18889: --- Assignee: Steve Loughran (was: Ahmar Suhail) > S3A: V2 SDK client does not work

[jira] [Commented] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766859#comment-17766859 ] Steve Loughran commented on HADOOP-18889: - and a few more to skip. one finding is that those

[jira] [Updated] (HADOOP-18576) Java 11 JavaDoc fails due to missing package comments

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18576: Fix Version/s: 3.3.9 > Java 11 JavaDoc fails due to missing package comments >

[jira] [Updated] (HADOOP-18890) remove okhttp usage

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18890: Fix Version/s: 3.4.0 > remove okhttp usage > --- > >

[jira] [Updated] (HADOOP-18890) remove okhttp usage

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18890: Issue Type: Improvement (was: Bug) > remove okhttp usage > --- > >

[jira] [Updated] (HADOOP-18890) remove okhttp usage

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18890: Component/s: build > remove okhttp usage > --- > > Key:

[jira] [Assigned] (HADOOP-18890) remove okhttp usage

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18890: --- Assignee: PJ Fanning > remove okhttp usage > --- > >

[jira] [Resolved] (HADOOP-18895) upgrade to commons-compress 1.24.0 due to CVE

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18895. - Fix Version/s: 3.3.9 Resolution: Fixed > upgrade to commons-compress 1.24.0 due

[jira] [Commented] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766751#comment-17766751 ] Steve Loughran commented on HADOOP-18889: - HADOOP-18908 to cover region stuff, this is for

[jira] [Assigned] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18908: --- Assignee: Ahmar Suhail > Improve s3a region handling, including determining from

[jira] [Commented] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

2023-09-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766547#comment-17766547 ] Steve Loughran commented on HADOOP-18908: - and a way to turn it off, even for aws buckets. >

[jira] [Updated] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

2023-09-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18908: Description: s3a now requires the fs.s3a.endpoint.region to be set; and while it can

[jira] [Updated] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

2023-09-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18908: Affects Version/s: 3.4.0 > Improve s3a region handling, including determining from

[jira] [Created] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

2023-09-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18908: --- Summary: Improve s3a region handling, including determining from endpoint Key: HADOOP-18908 URL: https://issues.apache.org/jira/browse/HADOOP-18908 Project:

[jira] [Updated] (HADOOP-18673) AWS SDK V2 - Refactor getS3Region & other follow up items

2023-09-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18673: Description: * Factor getS3Region into its own ExecutingStoreOperation; * Fix issue

[jira] [Created] (HADOOP-18907) ITestS3APrefetchingCacheFiles teardown failure if setup() fails

2023-09-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18907: --- Summary: ITestS3APrefetchingCacheFiles teardown failure if setup() fails Key: HADOOP-18907 URL: https://issues.apache.org/jira/browse/HADOOP-18907 Project:

<    1   2   3   4   5   6   7   8   9   10   >