[jira] [Updated] (HADOOP-18987) Corrections to Hadoop FileSystem API Definition

2023-11-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18987: Affects Version/s: 3.3.6 > Corrections to Hadoop FileSystem API Definition >

[jira] [Resolved] (HADOOP-18969) S3A: AbstractS3ACostTest to clear bucket fs.s3a.create.performance flag

2023-11-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18969. - Fix Version/s: 3.4.0 Resolution: Fixed > S3A: AbstractS3ACostTest to clear

[jira] [Commented] (HADOOP-18984) S3A: Support dynamic region resolution

2023-11-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788407#comment-17788407 ] Steve Loughran commented on HADOOP-18984: - fyi, cloudstore now has a regions2 command to work

[jira] [Created] (HADOOP-18984) S3A: Support dynamic region resolution

2023-11-20 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18984: --- Summary: S3A: Support dynamic region resolution Key: HADOOP-18984 URL: https://issues.apache.org/jira/browse/HADOOP-18984 Project: Hadoop Common Issue

[jira] [Created] (HADOOP-18983) S3A: remove @deprecated tags where no longer needed

2023-11-20 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18983: --- Summary: S3A: remove @deprecated tags where no longer needed Key: HADOOP-18983 URL: https://issues.apache.org/jira/browse/HADOOP-18983 Project: Hadoop Common

[jira] [Commented] (HADOOP-18980) S3A credential provider remapping: make extensible

2023-11-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787914#comment-17787914 ] Steve Loughran commented on HADOOP-18980: - exactly; though i'd expect the remapping to be from

[jira] [Created] (HADOOP-18980) S3A credential provider remapping: make extensible

2023-11-17 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18980: --- Summary: S3A credential provider remapping: make extensible Key: HADOOP-18980 URL: https://issues.apache.org/jira/browse/HADOOP-18980 Project: Hadoop Common

[jira] [Updated] (HADOOP-18830) S3A: Cut S3 Select

2023-11-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18830: Summary: S3A: Cut S3 Select (was: Cut S3 Select) > S3A: Cut S3 Select >

[jira] [Updated] (HADOOP-18975) AWS SDK v2: extend support for FIPS endpoints

2023-11-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18975: Summary: AWS SDK v2: extend support for FIPS endpoints (was: AWS SDK v2: support FIPS

[jira] [Assigned] (HADOOP-18975) AWS SDK v2: extend support for FIPS endpoints

2023-11-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18975: --- Assignee: Steve Loughran > AWS SDK v2: extend support for FIPS endpoints >

[jira] [Created] (HADOOP-18975) AWS SDK v2: support FIPS endpoints

2023-11-15 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18975: --- Summary: AWS SDK v2: support FIPS endpoints Key: HADOOP-18975 URL: https://issues.apache.org/jira/browse/HADOOP-18975 Project: Hadoop Common Issue

[jira] [Resolved] (HADOOP-18872) ABFS: Misreporting Retry Count for Sub-sequential and Parallel Operations

2023-11-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18872. - Fix Version/s: 3.4.0 Resolution: Fixed > ABFS: Misreporting Retry Count for

[jira] [Updated] (HADOOP-18845) Add ability to configure ConnectionTTL of http connections while creating S3 Client.

2023-11-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18845: Description: The option fs.s3a.connection.ttl sets the maximum time an idle connection

[jira] [Commented] (HADOOP-18967) Allow secure mode to be enabled with no downtime

2023-11-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17785460#comment-17785460 ] Steve Loughran commented on HADOOP-18967: - this will need hdfs and yarn tickets to match,

[jira] [Updated] (HADOOP-18930) S3A: make fs.s3a.create.performance an option you can set for the entire bucket

2023-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18930: Summary: S3A: make fs.s3a.create.performance an option you can set for the entire bucket

[jira] [Created] (HADOOP-18969) S3A: AbstractS3ACostTest to clear bucket fs.s3a.create.performance flag

2023-11-10 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18969: --- Summary: S3A: AbstractS3ACostTest to clear bucket fs.s3a.create.performance flag Key: HADOOP-18969 URL: https://issues.apache.org/jira/browse/HADOOP-18969

[jira] [Resolved] (HADOOP-18930) make fs.s3a.create.performance an option you can set for the entire bucket

2023-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18930. - Fix Version/s: 3.4.0 Resolution: Fixed > make fs.s3a.create.performance an

[jira] [Updated] (HADOOP-18968) S3A FileSystem does not correctly list empty directories after HADOOP-17199

2023-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18968: Summary: S3A FileSystem does not correctly list empty directories after HADOOP-17199

[jira] [Commented] (HADOOP-18968) S3A FileSystem does not correctly ls empty directories after HADOOP-17199

2023-11-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784901#comment-17784901 ] Steve Loughran commented on HADOOP-18968: - hmm. I'd have expected that to show up in testing.

[jira] [Updated] (HADOOP-18966) s3a file rename does double HEAD or LIST on source file/dir

2023-11-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18966: Summary: s3a file rename does double HEAD or LIST on source file/dir (was: s3a file

[jira] [Created] (HADOOP-18966) s3a file rename does double HEAD or LIST on source fiile/dir

2023-11-09 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18966: --- Summary: s3a file rename does double HEAD or LIST on source fiile/dir Key: HADOOP-18966 URL: https://issues.apache.org/jira/browse/HADOOP-18966 Project: Hadoop

[jira] [Commented] (HADOOP-18965) ITestS3AHugeFilesEncryption failure

2023-11-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784151#comment-17784151 ] Steve Loughran commented on HADOOP-18965: - happens with per bucket encryption as the new check

[jira] [Commented] (HADOOP-18965) ITestS3AHugeFilesEncryption failure

2023-11-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784145#comment-17784145 ] Steve Loughran commented on HADOOP-18965: - {code} [ERROR] Tests run: 10, Failures: 2, Errors:

[jira] [Created] (HADOOP-18965) ITestS3AHugeFilesEncryption failure

2023-11-08 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18965: --- Summary: ITestS3AHugeFilesEncryption failure Key: HADOOP-18965 URL: https://issues.apache.org/jira/browse/HADOOP-18965 Project: Hadoop Common Issue

[jira] [Commented] (HADOOP-18850) Enable dual-layer server-side encryption with AWS KMS keys (DSSE-KMS)

2023-11-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784138#comment-17784138 ] Steve Loughran commented on HADOOP-18850: - hey, think this is breaking my test runs on a bucket

[jira] [Resolved] (HADOOP-18487) Make protobuf 2.5 an optional runtime dependency.

2023-11-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18487. - Resolution: Fixed > Make protobuf 2.5 an optional runtime dependency. >

[jira] [Updated] (HADOOP-18487) Make protobuf 2.5 an optional runtime dependency.

2023-11-06 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18487: Fix Version/s: 3.3.9 > Make protobuf 2.5 an optional runtime dependency. >

[jira] [Updated] (HADOOP-18797) Support Concurrent Writes With S3A Magic Committer

2023-11-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18797: Fix Version/s: 3.3.9 > Support Concurrent Writes With S3A Magic Committer >

[jira] [Commented] (HADOOP-18962) Upgrade kafka to 3.4.0

2023-11-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782539#comment-17782539 ] Steve Loughran commented on HADOOP-18962: - add ...component = build, set version to affected

[jira] [Updated] (HADOOP-18933) upgrade netty to 4.1.100 due to CVE

2023-11-02 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18933: Fix Version/s: 3.3.9 > upgrade netty to 4.1.100 due to CVE >

[jira] [Updated] (HADOOP-18850) Enable dual-layer server-side encryption with AWS KMS keys (DSSE-KMS)

2023-11-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18850: Affects Version/s: 3.4.0 > Enable dual-layer server-side encryption with AWS KMS keys

[jira] [Updated] (HADOOP-18850) Enable dual-layer server-side encryption with AWS KMS keys (DSSE-KMS)

2023-11-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18850: Fix Version/s: 3.4.0 Resolution: Fixed Status: Resolved (was: Patch

[jira] [Updated] (HADOOP-18961) S3A: add s3guard command "bucket" to create buckets

2023-10-31 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18961: Summary: S3A: add s3guard command "bucket" to create buckets (was: S3A: add s3guard

[jira] [Commented] (HADOOP-18961) S3A: add s3guard command "bucket" to create buckets

2023-10-31 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781388#comment-17781388 ] Steve Loughran commented on HADOOP-18961: - initial option will be -create; we can add a -delete

[jira] [Updated] (HADOOP-18961) S3A: add s3guard command "mkbucket"

2023-10-31 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18961: Summary: S3A: add s3guard command "mkbucket" (was: add s3a command "mkbucket") > S3A:

[jira] [Created] (HADOOP-18961) add s3a command "mkbucket"

2023-10-31 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18961: --- Summary: add s3a command "mkbucket" Key: HADOOP-18961 URL: https://issues.apache.org/jira/browse/HADOOP-18961 Project: Hadoop Common Issue Type:

[jira] [Commented] (HADOOP-18952) FsCommand Stat class set the timeZone"UTC", which is different from the machine's timeZone

2023-10-30 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781087#comment-17781087 ] Steve Loughran commented on HADOOP-18952: - I think we will have to leave this alone...those

[jira] [Updated] (HADOOP-18955) AWS SDK v2: add path capability probe "fs.s3a.capability.aws.v2"

2023-10-30 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18955: Summary: AWS SDK v2: add path capability probe "fs.s3a.capability.aws.v2" (was: AWS SDK

[jira] [Commented] (HADOOP-18938) S3A region logic to handle vpce and non standard endpoints

2023-10-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779995#comment-17779995 ] Steve Loughran commented on HADOOP-18938: - this is why ITestS3AEndpointRegion.testWithVPCE

[jira] [Updated] (HADOOP-18915) S3A client builder to use getTImeDuration() for HTTP timeouts

2023-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18915: Summary: S3A client builder to use getTImeDuration() for HTTP timeouts (was: HTTP

[jira] [Commented] (HADOOP-18915) HTTP timeouts are not set correctly

2023-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779616#comment-17779616 ] Steve Loughran commented on HADOOP-18915: - so HADOOP-18889 does switch from seconds to millis,

[jira] [Created] (HADOOP-18951) S3A third party: document "Certificate doesn't match"

2023-10-25 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18951: --- Summary: S3A third party: document "Certificate doesn't match" Key: HADOOP-18951 URL: https://issues.apache.org/jira/browse/HADOOP-18951 Project: Hadoop Common

[jira] [Resolved] (HADOOP-18948) S3A. Add option fs.s3a.directory.operations.purge.uploads to purge on rename/delete

2023-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18948. - Fix Version/s: 3.4.0 Release Note: S3A directory delete and rename will

[jira] [Assigned] (HADOOP-18933) upgrade netty to 4.1.100 due to CVE

2023-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18933: --- Assignee: PJ Fanning > upgrade netty to 4.1.100 due to CVE >

[jira] [Resolved] (HADOOP-18933) upgrade netty to 4.1.100 due to CVE

2023-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18933. - Resolution: Fixed > upgrade netty to 4.1.100 due to CVE >

[jira] [Updated] (HADOOP-18933) upgrade netty to 4.1.100 due to CVE

2023-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18933: Component/s: build Fix Version/s: 3.4.0 Description: follow up to

[jira] [Resolved] (HADOOP-18932) Upgrade AWS v2 SDK to 2.20.160 and v1 to 1.12.565

2023-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18932. - Fix Version/s: 3.4.0 Assignee: Steve Loughran Resolution: Fixed >

[jira] [Updated] (HADOOP-18932) Upgrade AWS v2 SDK to 2.20.160 and v1 to 1.12.565

2023-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18932: Summary: Upgrade AWS v2 SDK to 2.20.160 and v1 to 1.12.565 (was: upgrade AWS v1 and v2

[jira] [Updated] (HADOOP-18948) S3A. Add option fs.s3a.directory.operations.purge.uploads to purge on rename/delete

2023-10-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18948: Summary: S3A. Add option fs.s3a.directory.operations.purge.uploads to purge on

[jira] [Resolved] (HADOOP-18949) upgrade maven dependency plugin due to security issue

2023-10-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18949. - Fix Version/s: 3.4.0 Assignee: PJ Fanning Resolution: Fixed > upgrade

[jira] [Commented] (HADOOP-18948) S3A. Add option fs.s3a.directory.operations.purge.uploads to do that on rename/delete

2023-10-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17778824#comment-17778824 ] Steve Loughran commented on HADOOP-18948: - # surprised we didn't have this earlier' better than

[jira] [Created] (HADOOP-18948) S3A. Add option fs.s3a.directory.operations.purge.uploads to do that on rename/delete

2023-10-23 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18948: --- Summary: S3A. Add option fs.s3a.directory.operations.purge.uploads to do that on rename/delete Key: HADOOP-18948 URL: https://issues.apache.org/jira/browse/HADOOP-18948

[jira] [Resolved] (HADOOP-18939) NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry()

2023-10-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18939. - Fix Version/s: 3.4.0 Assignee: Steve Loughran Resolution: Fixed > NPE

[jira] [Resolved] (HADOOP-18945) S3A: IAMInstanceCredentialsProvider failing: Failed to load credentials from IMDS

2023-10-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18945. - Fix Version/s: 3.4.0 Resolution: Fixed > S3A: IAMInstanceCredentialsProvider

[jira] [Commented] (HADOOP-18862) [JDK17] MiniYarnClusters don't launch in hadoop-aws integration tests

2023-10-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1651#comment-1651 ] Steve Loughran commented on HADOOP-18862: - should be fixed now. i'll need to retest > [JDK17]

[jira] [Resolved] (HADOOP-18946) S3A: testMultiObjectExceptionFilledIn() assertion error

2023-10-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18946. - Fix Version/s: 3.4.0 Resolution: Fixed > S3A:

[jira] [Assigned] (HADOOP-18946) S3A: testMultiObjectExceptionFilledIn() assertion error

2023-10-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18946: --- Assignee: Steve Loughran > S3A: testMultiObjectExceptionFilledIn() assertion

[jira] [Created] (HADOOP-18946) S3A: testMultiObjectExceptionFilledIn() assertion error

2023-10-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18946: --- Summary: S3A: testMultiObjectExceptionFilledIn() assertion error Key: HADOOP-18946 URL: https://issues.apache.org/jira/browse/HADOOP-18946 Project: Hadoop

[jira] [Commented] (HADOOP-18939) NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry()

2023-10-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776741#comment-17776741 ] Steve Loughran commented on HADOOP-18939: - aah, added a new failure from an assert that is

[jira] [Commented] (HADOOP-18945) S3A: IAMInstanceCredentialsProvider failing: Failed to load credentials from IMDS

2023-10-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776607#comment-17776607 ] Steve Loughran commented on HADOOP-18945: - looking for that error string "Failed to load

[jira] [Updated] (HADOOP-18945) S3A: IAMInstanceCredentialsProvider failing: Failed to load credentials from IMDS

2023-10-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18945: Description: Failures in impala test VMs using iAM for auth {code} Failed to open file

[jira] [Updated] (HADOOP-18945) S3A: IAMInstanceCredentialsProvider failing: Failed to load credentials from IMDS

2023-10-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18945: Summary: S3A: IAMInstanceCredentialsProvider failing: Failed to load credentials from

[jira] [Created] (HADOOP-18945) S3A: IAMInstanceCredentialsProvider failing: NoAwsCredentialsException: +: Failed to load credentials from IMDS.{noformat}

2023-10-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18945: --- Summary: S3A: IAMInstanceCredentialsProvider failing: NoAwsCredentialsException: +: Failed to load credentials from IMDS.{noformat} Key: HADOOP-18945 URL:

[jira] [Commented] (HADOOP-18944) s3a openfile is using readahead rather than async drain threshold to calculate drain threshold

2023-10-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776591#comment-17776591 ] Steve Loughran commented on HADOOP-18944: - one line fix and I don't even think we need a test

[jira] [Updated] (HADOOP-18944) s3a openfile is using readahead rather than async drain threshold to calculate drain threshold

2023-10-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18944: Description: we are using the wrong default value for the async drain threshold in

[jira] [Created] (HADOOP-18944) s3a openfile is using readahead rather than async drain threshold to calculate drain threshold

2023-10-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18944: --- Summary: s3a openfile is using readahead rather than async drain threshold to calculate drain threshold Key: HADOOP-18944 URL:

[jira] [Commented] (HADOOP-18938) S3A region logic to handle vpce and non standard endpoints

2023-10-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776243#comment-17776243 ] Steve Loughran commented on HADOOP-18938: - fail fast may not be the right strategy as the

[jira] [Resolved] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

2023-10-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18908. - Fix Version/s: 3.4.0 Resolution: Fixed > Improve s3a region handling, including

[jira] [Updated] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

2023-10-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18908: Description: S3A region logic improved for better inference and to be compatible with

[jira] [Created] (HADOOP-18940) ABFS: Remove commons IOUtils.close() from AbfsOutputStream

2023-10-17 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18940: --- Summary: ABFS: Remove commons IOUtils.close() from AbfsOutputStream Key: HADOOP-18940 URL: https://issues.apache.org/jira/browse/HADOOP-18940 Project: Hadoop

[jira] [Commented] (HADOOP-18938) S3A region logic to handle vpce and non standard endpoints

2023-10-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775842#comment-17775842 ] Steve Loughran commented on HADOOP-18938: - followup to HADOOP-18908 > S3A region logic to

[jira] [Updated] (HADOOP-18938) S3A region logic to handle vpce and non standard endpoints

2023-10-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18938: Summary: S3A region logic to handle vpce and non standard endpoints (was: Handle non

[jira] [Commented] (HADOOP-18939) NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry()

2023-10-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775773#comment-17775773 ] Steve Loughran commented on HADOOP-18939: - i can replicate this with MultiObjectDeleteException

[jira] [Commented] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from

2023-10-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775703#comment-17775703 ] Steve Loughran commented on HADOOP-18931: - this is service loader only, so for those where we

[jira] [Created] (HADOOP-18939) NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry()

2023-10-16 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18939: --- Summary: NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry() Key: HADOOP-18939 URL: https://issues.apache.org/jira/browse/HADOOP-18939 Project: Hadoop

[jira] [Updated] (HADOOP-18487) protobuf-2.5.0 dependencies => provided

2023-10-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18487: Fix Version/s: 3.4.0 > protobuf-2.5.0 dependencies => provided >

[jira] [Assigned] (HADOOP-18487) Make protobuf 2.5 an optional runtime dependency.

2023-10-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18487: --- Assignee: Steve Loughran > Make protobuf 2.5 an optional runtime dependency. >

[jira] [Updated] (HADOOP-18487) Make protobuf 2.5 an optional runtime dependency.

2023-10-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18487: Summary: Make protobuf 2.5 an optional runtime dependency. (was: protobuf-2.5.0

[jira] [Commented] (HADOOP-18487) protobuf-2.5.0 dependencies => provided

2023-10-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774902#comment-17774902 ] Steve Loughran commented on HADOOP-18487: - Part 1 PR of this is in trunk and up for 3.3: it is

[jira] [Resolved] (HADOOP-18890) remove okhttp usage

2023-10-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18890. - Fix Version/s: 3.3.9 Resolution: Fixed merged and backported. [~mukund-thakur]

[jira] [Resolved] (HADOOP-18857) AWS v2 SDK: fail meaningfully if legacy v2 signing requested

2023-10-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18857. - Fix Version/s: 3.4.0 Resolution: Fixed > AWS v2 SDK: fail meaningfully if

[jira] [Resolved] (HADOOP-18927) S3ARetryHandler to treat SocketExceptions as connectivity failures

2023-10-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18927. - Fix Version/s: 3.4.0 Resolution: Fixed > S3ARetryHandler to treat

[jira] [Created] (HADOOP-18935) coalesce AWS S3 client proxy settings

2023-10-12 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18935: --- Summary: coalesce AWS S3 client proxy settings Key: HADOOP-18935 URL: https://issues.apache.org/jira/browse/HADOOP-18935 Project: Hadoop Common Issue

[jira] [Commented] (HADOOP-18915) HTTP timeouts are not set correctly

2023-10-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774416#comment-17774416 ] Steve Loughran commented on HADOOP-18915: - actually there's more than the patch One of the

[jira] [Assigned] (HADOOP-18927) S3ARetryHandler to treat SocketExceptions as connectivity failures

2023-10-11 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18927: --- Assignee: Steve Loughran > S3ARetryHandler to treat SocketExceptions as

[jira] [Created] (HADOOP-18932) upgrade AWS v1 and v2 SDKs

2023-10-11 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18932: --- Summary: upgrade AWS v1 and v2 SDKs Key: HADOOP-18932 URL: https://issues.apache.org/jira/browse/HADOOP-18932 Project: Hadoop Common Issue Type:

[jira] [Created] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from

2023-10-11 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18931: --- Summary: FileSystem.getFileSystemClass() to log at debug the jar the .class came from Key: HADOOP-18931 URL: https://issues.apache.org/jira/browse/HADOOP-18931

[jira] [Commented] (HADOOP-18930) make fs.s3a.create.performance an option you can set for the entire bucket

2023-10-11 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774038#comment-17774038 ] Steve Loughran commented on HADOOP-18930: - thinking of making the path capability probe show

[jira] [Created] (HADOOP-18930) make fs.s3a.create.performance an option you can set for the entire bucket

2023-10-10 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18930: --- Summary: make fs.s3a.create.performance an option you can set for the entire bucket Key: HADOOP-18930 URL: https://issues.apache.org/jira/browse/HADOOP-18930

[jira] [Resolved] (HADOOP-18747) AWS SDK V2 - sigv2 support

2023-10-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18747. - Hadoop Flags: Incompatible change Release Note: AWS v2 signing is no longer

[jira] [Assigned] (HADOOP-18915) HTTP timeouts are not set correctly

2023-10-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18915: --- Assignee: Steve Loughran > HTTP timeouts are not set correctly >

[jira] [Commented] (HADOOP-18915) HTTP timeouts are not set correctly

2023-10-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773736#comment-17773736 ] Steve Loughran commented on HADOOP-18915: - let me fix in HADOOP-18889; it may have been part of

[jira] [Commented] (HADOOP-18927) S3ARetryHandler to treat SocketExceptions as connectivity failures

2023-10-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773628#comment-17773628 ] Steve Loughran commented on HADOOP-18927: - * treat NoRouteToHostException and BindException as

[jira] [Commented] (HADOOP-18927) S3ARetryHandler to treat SocketExceptions as connectivity failures

2023-10-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773622#comment-17773622 ] Steve Loughran commented on HADOOP-18927: - full v1 stack. I know this is v1, but v2 is where we

[jira] [Created] (HADOOP-18927) S3ARetryHandler to treat SocketExceptions as connectivity failures

2023-10-10 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18927: --- Summary: S3ARetryHandler to treat SocketExceptions as connectivity failures Key: HADOOP-18927 URL: https://issues.apache.org/jira/browse/HADOOP-18927 Project:

[jira] [Updated] (HADOOP-18869) ABFS: Fixing Behavior of a File System APIs on root path

2023-10-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18869: Fix Version/s: 3.4.0 > ABFS: Fixing Behavior of a File System APIs on root path >

[jira] [Resolved] (HADOOP-18861) ABFS: Fix failing CPK tests on trunk

2023-10-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18861. - Fix Version/s: 3.4.0 Resolution: Fixed > ABFS: Fix failing CPK tests on trunk >

[jira] [Commented] (HADOOP-18872) ABFS: Misreporting Retry Count for Sub-sequential and Parallel Operations

2023-10-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773406#comment-17773406 ] Steve Loughran commented on HADOOP-18872: - [~asrani_anmol] can you leave that "fix version"

[jira] [Updated] (HADOOP-18872) ABFS: Misreporting Retry Count for Sub-sequential and Parallel Operations

2023-10-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18872: Fix Version/s: (was: 3.3.6) > ABFS: Misreporting Retry Count for Sub-sequential and

[jira] [Resolved] (HADOOP-18876) ABFS: Change default from disk to bytebuffer for fs.azure.data.blocks.buffer

2023-10-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18876. - Fix Version/s: 3.4.0 (was: 3.3.6) Release Note: The

<    1   2   3   4   5   6   7   8   9   10   >