[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815951#comment-17815951 ] ASF GitHub Bot commented on HADOOP-19050: - adnanhemani commented on PR #6544: URL:

Re: [PR] HADOOP-19050. Add S3 Access Grants Support in S3A [hadoop]

2024-02-08 Thread via GitHub
adnanhemani commented on PR #6544: URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1935468938 WRT ITest failures, this is the `auth-keys.xml` file I'm using as per the suggestions on #6515. But I'm still getting the above failures - should this not be the case? Tested

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815946#comment-17815946 ] ASF GitHub Bot commented on HADOOP-19050: - adnanhemani commented on PR #6507: URL:

Re: [PR] HADOOP-19050, Add Support for AWS S3 Access Grants [hadoop]

2024-02-08 Thread via GitHub
adnanhemani commented on PR #6507: URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1935463310 This has been evolved into #6544. This PR will no longer be used now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815944#comment-17815944 ] ASF GitHub Bot commented on HADOOP-19050: - adnanhemani commented on PR #6544: URL:

Re: [PR] HADOOP-19050. Add S3 Access Grants Support in S3A [hadoop]

2024-02-08 Thread via GitHub
adnanhemani commented on PR #6544: URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1935462396 Hi @ahmarsuhail and @steveloughran - I've opened this with the new code that has the S3 Access Grants plugin as part of the AWS Java SDK bundle (this is an evolution of #6507). We

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815941#comment-17815941 ] ASF GitHub Bot commented on HADOOP-19050: - adnanhemani commented on PR #6544: URL:

Re: [PR] HADOOP-19050. Add S3 Access Grants Support in S3A [hadoop]

2024-02-08 Thread via GitHub
adnanhemani commented on PR #6544: URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1935460439 Unit tests and Tests results below, per my understanding all failures are due to various other known reasons: ``` ^C88665a0afd7e:hadoop-aws ahemani$ mvn verify -Dparallel-tests

[jira] [Commented] (HADOOP-19072) S3A: expand optimisations on stores with "fs.s3a.create.performance"

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815929#comment-17815929 ] ASF GitHub Bot commented on HADOOP-19072: - hadoop-yetus commented on PR #6543: URL:

Re: [PR] HADOOP-19072. S3A: expand optimisations on stores with "fs.s3a.create.performance" [hadoop]

2024-02-08 Thread via GitHub
hadoop-yetus commented on PR #6543: URL: https://github.com/apache/hadoop/pull/6543#issuecomment-1935424401 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815921#comment-17815921 ] ASF GitHub Bot commented on HADOOP-19050: - adnanhemani opened a new pull request, #6544: URL:

[PR] HADOOP-19050. Add S3 Access Grants Support in S3A [hadoop]

2024-02-08 Thread via GitHub
adnanhemani opened a new pull request, #6544: URL: https://github.com/apache/hadoop/pull/6544 ### Description of PR This adds S3 Access Grants support in S3A using the S3 Access Grants plugin, which is a part of the AWS Java SDK bundle starting in v2.23.19. As part of this PR,

Re: [PR] HDFS-17376. Distcp creates Factor 1 replication file on target if Source is EC. [hadoop]

2024-02-08 Thread via GitHub
sadanand48 commented on PR #6540: URL: https://github.com/apache/hadoop/pull/6540#issuecomment-1935358055 @sodonnel Could you please take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (HADOOP-19072) S3A: expand optimisations on stores with "fs.s3a.create.performance"

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815903#comment-17815903 ] ASF GitHub Bot commented on HADOOP-19072: - hadoop-yetus commented on PR #6543: URL:

Re: [PR] HADOOP-19072. S3A: expand optimisations on stores with "fs.s3a.create.performance" [hadoop]

2024-02-08 Thread via GitHub
hadoop-yetus commented on PR #6543: URL: https://github.com/apache/hadoop/pull/6543#issuecomment-1935301858 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Updated] (HADOOP-19072) S3A: expand optimisations on stores with "fs.s3a.create.performance"

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-19072: Labels: pull-request-available (was: ) > S3A: expand optimisations on stores with

[jira] [Commented] (HADOOP-19072) S3A: expand optimisations on stores with "fs.s3a.create.performance"

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815888#comment-17815888 ] ASF GitHub Bot commented on HADOOP-19072: - virajjasani opened a new pull request, #6543: URL:

Re: [PR] HADOOP-18508. Support multiple s3a integration test runs on same bucket in parallel [hadoop]

2024-02-08 Thread via GitHub
hadoop-yetus commented on PR #5081: URL: https://github.com/apache/hadoop/pull/5081#issuecomment-1935201442 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18508) support multiple s3a integration test runs on same bucket in parallel

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815886#comment-17815886 ] ASF GitHub Bot commented on HADOOP-18508: - hadoop-yetus commented on PR #5081: URL:

[jira] [Updated] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-02-08 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HADOOP-19066: -- Status: Patch Available (was: In Progress) > AWS SDK V2 - Enabling FIPS should be allowed

Re: [PR] HADOOP-19066. S3A: AWS SDK V2 - Enabling FIPS should be allowed with central endpoint [hadoop]

2024-02-08 Thread via GitHub
virajjasani commented on PR #6539: URL: https://github.com/apache/hadoop/pull/6539#issuecomment-1935170066 Tested against `us-west-2` bucket with endpoints: `s3.amazonaws.com` and `s3-us-west-2.amazonaws.com`: `mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Dscale

[jira] [Commented] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815878#comment-17815878 ] ASF GitHub Bot commented on HADOOP-19066: - virajjasani commented on PR #6539: URL:

[jira] [Commented] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.2

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815873#comment-17815873 ] ASF GitHub Bot commented on HADOOP-19071: - hadoop-yetus commented on PR #6537: URL:

Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]

2024-02-08 Thread via GitHub
hadoop-yetus commented on PR #6537: URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1935142238 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19012) Use CRC tables to speed up galoisFieldMultiply in CrcUtil

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815869#comment-17815869 ] ASF GitHub Bot commented on HADOOP-19012: - hadoop-yetus commented on PR #6542: URL:

Re: [PR] HADOOP-19012. Use CRC tables to speed up galoisFieldMultiply in CrcUtil. [hadoop]

2024-02-08 Thread via GitHub
hadoop-yetus commented on PR #6542: URL: https://github.com/apache/hadoop/pull/6542#issuecomment-1935111437 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.2

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815860#comment-17815860 ] ASF GitHub Bot commented on HADOOP-19071: - hadoop-yetus commented on PR #6537: URL:

Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]

2024-02-08 Thread via GitHub
hadoop-yetus commented on PR #6537: URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1935080767 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at

[jira] [Commented] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.2

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815858#comment-17815858 ] ASF GitHub Bot commented on HADOOP-19071: - slfan1989 commented on PR #6537: URL:

Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]

2024-02-08 Thread via GitHub
slfan1989 commented on PR #6537: URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1935076896 @sjlee @steveloughran Thank you for your attention to this PR! When I chose this version, I checked the dependencies of the Spark and Flink projects. As of now, Spark is using 3.1.2,

Re: [PR] HADOOP-19069. Use hadoop-thirdparty 1.2.0. (#6533) [hadoop]

2024-02-08 Thread via GitHub
hadoop-yetus commented on PR #6541: URL: https://github.com/apache/hadoop/pull/6541#issuecomment-1935025565 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19069) Use hadoop-thirdparty 1.2.0

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815846#comment-17815846 ] ASF GitHub Bot commented on HADOOP-19069: - hadoop-yetus commented on PR #6541: URL:

[jira] [Updated] (HADOOP-19012) Use CRC tables to speed up galoisFieldMultiply in CrcUtil

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-19012: Labels: pull-request-available (was: ) > Use CRC tables to speed up galoisFieldMultiply

[jira] [Commented] (HADOOP-19012) Use CRC tables to speed up galoisFieldMultiply in CrcUtil

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815839#comment-17815839 ] ASF GitHub Bot commented on HADOOP-19012: - szetszwo opened a new pull request, #6542: URL:

[jira] [Resolved] (HADOOP-19059) update AWS SDK to support S3 Access Grants in S3A

2024-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19059. - Fix Version/s: 3.5.0 3.4.1 Resolution: Fixed > update AWS

[jira] [Updated] (HADOOP-19059) S3A: update AWS SDK to 2.23.19 to support S3 Access Grants

2024-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19059: Summary: S3A: update AWS SDK to 2.23.19 to support S3 Access Grants (was: update AWS

[jira] [Commented] (HADOOP-19059) update AWS SDK to support S3 Access Grants in S3A

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815829#comment-17815829 ] ASF GitHub Bot commented on HADOOP-19059: - steveloughran merged PR #6538: URL:

Re: [PR] HADOOP-19059. Update AWS Java SDK to 2.23.19 [hadoop]

2024-02-08 Thread via GitHub
steveloughran merged PR #6538: URL: https://github.com/apache/hadoop/pull/6538 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Updated] (HADOOP-18830) S3A: Cut S3 Select

2024-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18830: Fix Version/s: 3.3.9 > S3A: Cut S3 Select > -- > > Key:

Re: [PR] HADOOP-18830. Cut S3 Select (#6144) [hadoop]

2024-02-08 Thread via GitHub
steveloughran merged PR #6523: URL: https://github.com/apache/hadoop/pull/6523 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18830) S3A: Cut S3 Select

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815827#comment-17815827 ] ASF GitHub Bot commented on HADOOP-18830: - steveloughran merged PR #6523: URL:

[jira] [Assigned] (HADOOP-19072) S3A: expand optimisations on stores with "fs.s3a.create.performance"

2024-02-08 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani reassigned HADOOP-19072: - Assignee: Viraj Jasani > S3A: expand optimisations on stores with

[jira] [Commented] (HADOOP-19059) update AWS SDK to support S3 Access Grants in S3A

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815825#comment-17815825 ] ASF GitHub Bot commented on HADOOP-19059: - steveloughran commented on PR #6538: URL:

Re: [PR] HADOOP-19059. Update AWS Java SDK to 2.23.19 [hadoop]

2024-02-08 Thread via GitHub
steveloughran commented on PR #6538: URL: https://github.com/apache/hadoop/pull/6538#issuecomment-1934886426 thanks. look at #6515 for some settings you can add to your auth-keys file to stop most of the landsat failures; this is how we will have to work around the failures on old

Re: [PR] HDFS-17376. Distcp creates Factor 1 replication file on target if Source is EC. [hadoop]

2024-02-08 Thread via GitHub
hadoop-yetus commented on PR #6540: URL: https://github.com/apache/hadoop/pull/6540#issuecomment-1934876888 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19072) S3A: expand optimisations on stores with "fs.s3a.create.performance"

2024-02-08 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815822#comment-17815822 ] Viraj Jasani commented on HADOOP-19072: --- The improvement makes sense, as long as downstreamer

[jira] [Updated] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.2

2024-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19071: Component/s: build > Update maven-surefire-plugin from 3.0.0 to 3.2.2 >

[jira] [Commented] (HADOOP-19069) Use hadoop-thirdparty 1.2.0

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815797#comment-17815797 ] ASF GitHub Bot commented on HADOOP-19069: - steveloughran commented on PR #6533: URL:

Re: [PR] HADOOP-19069. Use hadoop-thirdparty 1.2.0. [hadoop]

2024-02-08 Thread via GitHub
steveloughran commented on PR #6533: URL: https://github.com/apache/hadoop/pull/6533#issuecomment-1934747233 doing a backport to branch-3.3; let's see what yetus says. If it is happy I will merge there too -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (HADOOP-19069) Use hadoop-thirdparty 1.2.0

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815796#comment-17815796 ] ASF GitHub Bot commented on HADOOP-19069: - steveloughran opened a new pull request, #6541: URL:

[PR] HADOOP-19069. Use hadoop-thirdparty 1.2.0. (#6533) [hadoop]

2024-02-08 Thread via GitHub
steveloughran opened a new pull request, #6541: URL: https://github.com/apache/hadoop/pull/6541 PR #6533 applied to branch-3.3 Contributed by Shilun Fan Reviewed-by: He Xiaoqiao ### How was this patch tested? ### For code changes: - [ ] Does

[PR] HDFS-17376. Distcp creates Factor 1 replication file on target if Source is EC. [hadoop]

2024-02-08 Thread via GitHub
sadanand48 opened a new pull request, #6540: URL: https://github.com/apache/hadoop/pull/6540 ### Description of PR If the source file is EC, distcp without preserve option creates a 1 replication file (this is not intended). This is because for an EC file getReplication() always

[jira] [Commented] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.2

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815774#comment-17815774 ] ASF GitHub Bot commented on HADOOP-19071: - sjlee commented on PR #6537: URL:

Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]

2024-02-08 Thread via GitHub
sjlee commented on PR #6537: URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1934674117 Out of curiosity, what was the reason to pick 3.2.2 as the version to upgrade? The latest version seems to be 3.2.5 and there seem to be quite a few fixes between these two versions. --

[jira] [Commented] (HADOOP-19059) update AWS SDK to support S3 Access Grants in S3A

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815767#comment-17815767 ] ASF GitHub Bot commented on HADOOP-19059: - hadoop-yetus commented on PR #6538: URL:

Re: [PR] HADOOP-19059. Update AWS Java SDK to 2.23.19 [hadoop]

2024-02-08 Thread via GitHub
hadoop-yetus commented on PR #6538: URL: https://github.com/apache/hadoop/pull/6538#issuecomment-1934574853 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Assigned] (HADOOP-19057) S3 public test bucket landsat-pds unreadable -needs replacement

2024-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-19057: --- Assignee: Steve Loughran > S3 public test bucket landsat-pds unreadable -needs

[jira] [Created] (HADOOP-19072) S3A: expand optimisations on stores with "fs.s3a.create.performance"

2024-02-08 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19072: --- Summary: S3A: expand optimisations on stores with "fs.s3a.create.performance" Key: HADOOP-19072 URL: https://issues.apache.org/jira/browse/HADOOP-19072

[jira] [Updated] (HADOOP-18886) S3A: AWS SDK V2 Migration: stabilization and S3Express

2024-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18886: Target Version/s: 3.4.1 (was: 3.5.0) > S3A: AWS SDK V2 Migration: stabilization and

Re: [PR] HDFS-17374. EC: StripedBlockReader#newConnectedPeer should set SO_TIMEOUT and SO_KEEPALIVE [hadoop]

2024-02-08 Thread via GitHub
hadoop-yetus commented on PR #6536: URL: https://github.com/apache/hadoop/pull/6536#issuecomment-1934283866 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Updated] (HADOOP-18996) S3A to provide full support for S3 Express One Zone

2024-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18996: Release Note: Hadoop S3A connector has explicit awareness of and support for S3Express

[jira] [Updated] (HADOOP-18996) S3A to provide full support for S3 Express One Zone

2024-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18996: Description: HADOOP-18995 upgrades the SDK version which allows connecting to a s3

Re: [PR] HDFS-17374. EC: StripedBlockReader#newConnectedPeer should set SO_TIMEOUT and SO_KEEPALIVE [hadoop]

2024-02-08 Thread via GitHub
hadoop-yetus commented on PR #6536: URL: https://github.com/apache/hadoop/pull/6536#issuecomment-1934254850 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Updated] (HADOOP-19045) HADOOP-19045. S3A: CreateSession Timeout after 10 seconds

2024-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19045: Description: s3a client timeout settings are getting down to http client, but not sdk

[jira] [Updated] (HADOOP-19045) HADOOP-19045. S3A: CreateSession Timeout after 10 seconds

2024-02-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19045: Summary: HADOOP-19045. S3A: CreateSession Timeout after 10 seconds (was: S3A: pass

[jira] [Commented] (HADOOP-19047) Support InMemory Tracking Of S3A Magic Commits

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815691#comment-17815691 ] ASF GitHub Bot commented on HADOOP-19047: - shameersss1 commented on PR #6468: URL:

Re: [PR] HADOOP-19047: Support InMemory Tracking Of S3A Magic Commits [hadoop]

2024-02-08 Thread via GitHub
shameersss1 commented on PR #6468: URL: https://github.com/apache/hadoop/pull/6468#issuecomment-1934214757 @steveloughran - Gentle reminder for review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Resolved] (HADOOP-19069) Use hadoop-thirdparty 1.2.0

2024-02-08 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan resolved HADOOP-19069. - Fix Version/s: 3.4.0 3.5.0 Hadoop Flags: Reviewed Resolution:

[jira] [Commented] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.2

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815638#comment-17815638 ] ASF GitHub Bot commented on HADOOP-19071: - slfan1989 commented on PR #6537: URL:

Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]

2024-02-08 Thread via GitHub
slfan1989 commented on PR #6537: URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1933872180 @steveloughran Can you help review this PR? Thank you very much! It looks like we can directly upgrade maven-surefire-plugin to 3.2.2. -- This is an automated message from the Apache

[jira] [Commented] (HADOOP-19069) Use hadoop-thirdparty 1.2.0

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815637#comment-17815637 ] ASF GitHub Bot commented on HADOOP-19069: - slfan1989 commented on PR #6533: URL:

[jira] [Commented] (HADOOP-19069) Use hadoop-thirdparty 1.2.0

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815636#comment-17815636 ] ASF GitHub Bot commented on HADOOP-19069: - slfan1989 merged PR #6533: URL:

Re: [PR] HADOOP-19069. Use hadoop-thirdparty 1.2.0. [hadoop]

2024-02-08 Thread via GitHub
slfan1989 commented on PR #6533: URL: https://github.com/apache/hadoop/pull/6533#issuecomment-1933866028 @Hexiaoqiao Thank you very much for the review PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] HADOOP-19069. Use hadoop-thirdparty 1.2.0. [hadoop]

2024-02-08 Thread via GitHub
slfan1989 merged PR #6533: URL: https://github.com/apache/hadoop/pull/6533 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815604#comment-17815604 ] ASF GitHub Bot commented on HADOOP-19066: - hadoop-yetus commented on PR #6539: URL:

Re: [PR] HADOOP-19066. S3A: AWS SDK V2 - Enabling FIPS should be allowed with central endpoint [hadoop]

2024-02-08 Thread via GitHub
hadoop-yetus commented on PR #6539: URL: https://github.com/apache/hadoop/pull/6539#issuecomment-1933723159 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17374. EC: StripedBlockReader#newConnectedPeer should set SO_TIMEOUT and SO_KEEPALIVE [hadoop]

2024-02-08 Thread via GitHub
hfutatzhanghb commented on PR #6536: URL: https://github.com/apache/hadoop/pull/6536#issuecomment-1933709426 @Hexiaoqiao @zhangshuyan0 @tomscut @tasanuma Sir, PTAL when you have free time, thanks a lot~ -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (HADOOP-14837) Handle S3A "glacier" data

2024-02-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815565#comment-17815565 ] ASF GitHub Bot commented on HADOOP-14837: - hadoop-yetus commented on PR #6407: URL:

Re: [PR] HADOOP-14837 : Support Read Restored Glacier Objects [hadoop]

2024-02-08 Thread via GitHub
hadoop-yetus commented on PR #6407: URL: https://github.com/apache/hadoop/pull/6407#issuecomment-1933545491 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |