[jira] [Commented] (HADOOP-18342) Upgrade to Avro 1.11.1

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816277#comment-17816277 ] ASF GitHub Bot commented on HADOOP-18342: - hadoop-yetus commented on PR #4854: URL:

Re: [PR] HADOOP-18342: shaded avro jar [hadoop]

2024-02-09 Thread via GitHub
hadoop-yetus commented on PR #4854: URL: https://github.com/apache/hadoop/pull/4854#issuecomment-1936868233 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19057) S3 public test bucket landsat-pds unreadable -needs replacement

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816271#comment-17816271 ] ASF GitHub Bot commented on HADOOP-19057: - virajjasani commented on PR #6515: URL:

Re: [PR] HADOOP-19057. S3A: Landsat bucket used in tests no longer accessible [hadoop]

2024-02-09 Thread via GitHub
virajjasani commented on PR #6515: URL: https://github.com/apache/hadoop/pull/6515#issuecomment-1936824678 @ahmarsuhail are you aware of any criteria used by Amazon to recycle public buckets? -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Commented] (HADOOP-19057) S3 public test bucket landsat-pds unreadable -needs replacement

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816269#comment-17816269 ] ASF GitHub Bot commented on HADOOP-19057: - virajjasani commented on PR #6515: URL:

Re: [PR] HADOOP-19057. S3A: Landsat bucket used in tests no longer accessible [hadoop]

2024-02-09 Thread via GitHub
virajjasani commented on PR #6515: URL: https://github.com/apache/hadoop/pull/6515#issuecomment-1936823636 Shall we not use requester pay public bucket for all landsat usages? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Commented] (HADOOP-19059) S3A: update AWS SDK to 2.23.19 to support S3 Access Grants

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816267#comment-17816267 ] ASF GitHub Bot commented on HADOOP-19059: - adnanhemani commented on PR #6506: URL:

Re: [PR] HADOOP-19059. Update AWS SDK to v2.23.7 [hadoop]

2024-02-09 Thread via GitHub
adnanhemani commented on PR #6506: URL: https://github.com/apache/hadoop/pull/6506#issuecomment-1936813337 @steveloughran please go ahead and close (I would, but GH doesn't allow me to) -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Commented] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.5

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816266#comment-17816266 ] ASF GitHub Bot commented on HADOOP-19071: - hadoop-yetus commented on PR #6545: URL:

Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]

2024-02-09 Thread via GitHub
hadoop-yetus commented on PR #6545: URL: https://github.com/apache/hadoop/pull/6545#issuecomment-1936809566 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816263#comment-17816263 ] ASF GitHub Bot commented on HADOOP-19050: - adnanhemani commented on PR #6544: URL:

Re: [PR] HADOOP-19050. Add S3 Access Grants Support in S3A [hadoop]

2024-02-09 Thread via GitHub
adnanhemani commented on PR #6544: URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1936795463 Hi @ahmarsuhail, I've made all changes requested. Please take a look when you can. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816261#comment-17816261 ] ASF GitHub Bot commented on HADOOP-19050: - hadoop-yetus commented on PR #6544: URL:

Re: [PR] HADOOP-19050. Add S3 Access Grants Support in S3A [hadoop]

2024-02-09 Thread via GitHub
hadoop-yetus commented on PR #6544: URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1936789090 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]

2024-02-09 Thread via GitHub
hadoop-yetus commented on PR #6545: URL: https://github.com/apache/hadoop/pull/6545#issuecomment-1936757691 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at

[jira] [Commented] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.5

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816256#comment-17816256 ] ASF GitHub Bot commented on HADOOP-19071: - slfan1989 opened a new pull request, #6545: URL:

[PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]

2024-02-09 Thread via GitHub
slfan1989 opened a new pull request, #6545: URL: https://github.com/apache/hadoop/pull/6545 ### Description of PR JIRA: HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. ### How was this patch tested? ### For code changes: - [ ] Does the

[jira] [Updated] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.5

2024-02-09 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan updated HADOOP-19071: Summary: Update maven-surefire-plugin from 3.0.0 to 3.2.5 (was: Update maven-surefire-plugin

[jira] [Updated] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.2

2024-02-09 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan updated HADOOP-19071: Target Version/s: 3.4.0, 3.5.0 (was: 3.5.0) > Update maven-surefire-plugin from 3.0.0 to 3.2.2

[jira] [Updated] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.2

2024-02-09 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan updated HADOOP-19071: Affects Version/s: 3.4.0 > Update maven-surefire-plugin from 3.0.0 to 3.2.2 >

[jira] [Commented] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.2

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816251#comment-17816251 ] ASF GitHub Bot commented on HADOOP-19071: - slfan1989 commented on PR #6537: URL:

Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]

2024-02-09 Thread via GitHub
slfan1989 commented on PR #6537: URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1936744038 > +1 for the change on trunk/3.4; let's see what surprises surface...we only need to worry about build time issues, not production ones. I will submit a pull request to trunk3.4

Re: [PR] HADOOP-19050. Add S3 Access Grants Support in S3A [hadoop]

2024-02-09 Thread via GitHub
adnanhemani commented on code in PR #6544: URL: https://github.com/apache/hadoop/pull/6544#discussion_r1484871939 ## hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md: ## @@ -614,6 +614,38 @@ If the following property is not set or set to `true`, the

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816240#comment-17816240 ] ASF GitHub Bot commented on HADOOP-19050: - adnanhemani commented on code in PR #6544: URL:

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816235#comment-17816235 ] ASF GitHub Bot commented on HADOOP-19050: - adnanhemani commented on code in PR #6544: URL:

Re: [PR] HADOOP-19050. Add S3 Access Grants Support in S3A [hadoop]

2024-02-09 Thread via GitHub
adnanhemani commented on code in PR #6544: URL: https://github.com/apache/hadoop/pull/6544#discussion_r1484857819 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AccessGrantConfiguration.java: ## @@ -0,0 +1,108 @@ +/* + * Licensed to the Apache Software

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816232#comment-17816232 ] ASF GitHub Bot commented on HADOOP-19050: - adnanhemani commented on code in PR #6544: URL:

Re: [PR] HADOOP-19050. Add S3 Access Grants Support in S3A [hadoop]

2024-02-09 Thread via GitHub
adnanhemani commented on code in PR #6544: URL: https://github.com/apache/hadoop/pull/6544#discussion_r1484852608 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AccessGrantConfiguration.java: ## @@ -0,0 +1,108 @@ +/* + * Licensed to the Apache Software

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816231#comment-17816231 ] ASF GitHub Bot commented on HADOOP-19050: - adnanhemani commented on code in PR #6544: URL:

Re: [PR] HADOOP-19050. Add S3 Access Grants Support in S3A [hadoop]

2024-02-09 Thread via GitHub
adnanhemani commented on code in PR #6544: URL: https://github.com/apache/hadoop/pull/6544#discussion_r1484851522 ## hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md: ## @@ -614,6 +614,38 @@ If the following property is not set or set to `true`, the

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816230#comment-17816230 ] ASF GitHub Bot commented on HADOOP-19050: - adnanhemani commented on code in PR #6544: URL:

Re: [PR] HADOOP-19050. Add S3 Access Grants Support in S3A [hadoop]

2024-02-09 Thread via GitHub
adnanhemani commented on code in PR #6544: URL: https://github.com/apache/hadoop/pull/6544#discussion_r1484848907 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.java: ## @@ -401,4 +411,19 @@ private static Region

[jira] [Commented] (HADOOP-19047) Support InMemory Tracking Of S3A Magic Commits

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816206#comment-17816206 ] ASF GitHub Bot commented on HADOOP-19047: - steveloughran commented on code in PR #6468: URL:

Re: [PR] HADOOP-19047: Support InMemory Tracking Of S3A Magic Commits [hadoop]

2024-02-09 Thread via GitHub
steveloughran commented on code in PR #6468: URL: https://github.com/apache/hadoop/pull/6468#discussion_r1484762085 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/commit/CommitConstants.java: ## @@ -242,6 +242,13 @@ private CommitConstants() { */

[jira] [Commented] (HADOOP-19047) Support InMemory Tracking Of S3A Magic Commits

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816205#comment-17816205 ] ASF GitHub Bot commented on HADOOP-19047: - steveloughran commented on code in PR #6468: URL:

Re: [PR] HADOOP-19047: Support InMemory Tracking Of S3A Magic Commits [hadoop]

2024-02-09 Thread via GitHub
steveloughran commented on code in PR #6468: URL: https://github.com/apache/hadoop/pull/6468#discussion_r1484756242 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/commit/magic/MagicS3GuardCommitter.java: ## @@ -264,9 +326,14 @@ public void

[jira] [Commented] (HADOOP-14837) Handle S3A "glacier" data

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816203#comment-17816203 ] ASF GitHub Bot commented on HADOOP-14837: - steveloughran commented on code in PR #6407: URL:

Re: [PR] HADOOP-14837 : Support Read Restored Glacier Objects [hadoop]

2024-02-09 Thread via GitHub
steveloughran commented on code in PR #6407: URL: https://github.com/apache/hadoop/pull/6407#discussion_r1484732749 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ObjectStorageClassFilter.java: ## @@ -27,19 +27,25 @@ /** * - * {@link

[jira] [Commented] (HADOOP-18938) S3A region logic to handle vpce and non standard endpoints

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816200#comment-17816200 ] ASF GitHub Bot commented on HADOOP-18938: - hadoop-yetus commented on PR #6466: URL:

Re: [PR] HADOOP-18938. AWS SDK v2: Fix endpoint region parsing for vpc endpoints. [hadoop]

2024-02-09 Thread via GitHub
hadoop-yetus commented on PR #6466: URL: https://github.com/apache/hadoop/pull/6466#issuecomment-1936513805 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] MAPREDUCE-7448. Inconsistent Behavior for FileOutputCommitter V1 to commit successfully many times [hadoop]

2024-02-09 Thread via GitHub
steveloughran commented on code in PR #6038: URL: https://github.com/apache/hadoop/pull/6038#discussion_r1484731817 ## hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java: ##

[jira] [Commented] (HADOOP-19073) WASB: Fix connection leak in FolderRenamePending

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816196#comment-17816196 ] ASF GitHub Bot commented on HADOOP-19073: - steveloughran commented on PR #6534: URL:

Re: [PR] HADOOP-19073 WASB: Fix connection leak in FolderRenamePending [hadoop]

2024-02-09 Thread via GitHub
steveloughran commented on PR #6534: URL: https://github.com/apache/hadoop/pull/6534#issuecomment-1936498628 @xuzifu666 yes. 1. Which azure region did you run all the wasb tests against 2. and what were your maven command line arguments used? 3. did any tests fail? -- This is an

[jira] [Commented] (HADOOP-18679) Add API for bulk/paged object deletion

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816171#comment-17816171 ] ASF GitHub Bot commented on HADOOP-18679: - hadoop-yetus commented on PR #6494: URL:

Re: [PR] HADOOP-18679. Add API for bulk/paged object deletion [hadoop]

2024-02-09 Thread via GitHub
hadoop-yetus commented on PR #6494: URL: https://github.com/apache/hadoop/pull/6494#issuecomment-1936382465 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17376. Distcp creates Factor 1 replication file on target if Source is EC. [hadoop]

2024-02-09 Thread via GitHub
sodonnel merged PR #6540: URL: https://github.com/apache/hadoop/pull/6540 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18938) S3A region logic to handle vpce and non standard endpoints

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816152#comment-17816152 ] ASF GitHub Bot commented on HADOOP-18938: - hadoop-yetus commented on PR #6466: URL:

Re: [PR] HADOOP-18938. AWS SDK v2: Fix endpoint region parsing for vpc endpoints. [hadoop]

2024-02-09 Thread via GitHub
hadoop-yetus commented on PR #6466: URL: https://github.com/apache/hadoop/pull/6466#issuecomment-1936281849 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17376. Distcp creates Factor 1 replication file on target if Source is EC. [hadoop]

2024-02-09 Thread via GitHub
hadoop-yetus commented on PR #6540: URL: https://github.com/apache/hadoop/pull/6540#issuecomment-1936233134 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18938) S3A region logic to handle vpce and non standard endpoints

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816110#comment-17816110 ] ASF GitHub Bot commented on HADOOP-18938: - shintaroonuma commented on PR #6466: URL:

Re: [PR] HADOOP-18938. AWS SDK v2: Fix endpoint region parsing for vpc endpoints. [hadoop]

2024-02-09 Thread via GitHub
shintaroonuma commented on PR #6466: URL: https://github.com/apache/hadoop/pull/6466#issuecomment-1936098571 Thanks for the comments, rebased and updated the pr. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Commented] (HADOOP-18938) S3A region logic to handle vpce and non standard endpoints

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816108#comment-17816108 ] ASF GitHub Bot commented on HADOOP-18938: - shintaroonuma commented on code in PR #6466: URL:

[jira] [Commented] (HADOOP-18938) S3A region logic to handle vpce and non standard endpoints

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816109#comment-17816109 ] ASF GitHub Bot commented on HADOOP-18938: - shintaroonuma commented on code in PR #6466: URL:

Re: [PR] HADOOP-18938. AWS SDK v2: Fix endpoint region parsing for vpc endpoints. [hadoop]

2024-02-09 Thread via GitHub
shintaroonuma commented on code in PR #6466: URL: https://github.com/apache/hadoop/pull/6466#discussion_r148513 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.java: ## @@ -82,6 +84,9 @@ public class DefaultS3ClientFactory extends

Re: [PR] HADOOP-18938. AWS SDK v2: Fix endpoint region parsing for vpc endpoints. [hadoop]

2024-02-09 Thread via GitHub
shintaroonuma commented on code in PR #6466: URL: https://github.com/apache/hadoop/pull/6466#discussion_r148232 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.java: ## @@ -361,7 +366,15 @@ private static URI getS3Endpoint(String

Re: [PR] HADOOP-19050. Add S3 Access Grants Support in S3A [hadoop]

2024-02-09 Thread via GitHub
ahmarsuhail commented on PR #6544: URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1936082879 @adnanhemani re test failures, just updating the core-site won't be enough for some of them, you'll also need the code changes in Steve's PR https://github.com/apache/hadoop/pull/6515

[jira] [Commented] (HADOOP-19057) S3 public test bucket landsat-pds unreadable -needs replacement

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816105#comment-17816105 ] ASF GitHub Bot commented on HADOOP-19057: - ahmarsuhail commented on code in PR #6515: URL:

Re: [PR] HADOOP-19057. S3A: Landsat bucket used in tests no longer accessible [hadoop]

2024-02-09 Thread via GitHub
ahmarsuhail commented on code in PR #6515: URL: https://github.com/apache/hadoop/pull/6515#discussion_r1484407197 ## hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/connecting.md: ## @@ -289,9 +289,8 @@ for buckets in the central and EU/Ireland endpoints. ```xml

[jira] [Commented] (HADOOP-19073) WASB: Fix connection leak in FolderRenamePending

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816093#comment-17816093 ] ASF GitHub Bot commented on HADOOP-19073: - xuzifu666 commented on PR #6534: URL:

Re: [PR] HADOOP-19073 WASB: Fix connection leak in FolderRenamePending [hadoop]

2024-02-09 Thread via GitHub
xuzifu666 commented on PR #6534: URL: https://github.com/apache/hadoop/pull/6534#issuecomment-1936056092 > moved to hadoop; azure component. > > now, the bad news: as it goes near a cloud store, you *have* to run the entire hadoop-azure tests for the wasb component, and tell us

[jira] [Commented] (HADOOP-19073) WASB: Fix connection leak in FolderRenamePending

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816088#comment-17816088 ] ASF GitHub Bot commented on HADOOP-19073: - steveloughran commented on PR #6534: URL:

Re: [PR] HADOOP-19073 WASB: Fix connection leak in FolderRenamePending [hadoop]

2024-02-09 Thread via GitHub
steveloughran commented on PR #6534: URL: https://github.com/apache/hadoop/pull/6534#issuecomment-1936045642 moved to hadoop; azure component. now, the bad news: as it goes near a cloud store, you *have* to run the entire hadoop-azure tests for the wasb component, and tell us which

[jira] [Updated] (HADOOP-19073) WASB: Fix connection leak in FolderRenamePending

2024-02-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19073: Summary: WASB: Fix connection leak in FolderRenamePending (was: Fix connection leak in

[jira] [Assigned] (HADOOP-19073) Fix connection leak in FolderRenamePending

2024-02-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-19073: --- Key: HADOOP-19073 (was: HDFS-17373) Affects Version/s: 3.3.6

[jira] [Commented] (HADOOP-18980) S3A credential provider remapping: make extensible

2024-02-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816085#comment-17816085 ] Steve Loughran commented on HADOOP-18980: - as commented in the backport pr, there are some

[jira] [Updated] (HADOOP-19073) Fix connection leak in FolderRenamePending

2024-02-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19073: Component/s: fs/azure > Fix connection leak in FolderRenamePending >

[jira] [Updated] (HADOOP-18980) S3A credential provider remapping: make extensible

2024-02-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18980: Fix Version/s: 3.4.1 > S3A credential provider remapping: make extensible >

Re: [PR] HADOOP-18980. S3A credential provider remapping: make extensible (#6406) [hadoop]

2024-02-09 Thread via GitHub
steveloughran merged PR #6525: URL: https://github.com/apache/hadoop/pull/6525 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HADOOP-18843. Guava version 32.0.1 bump to fix CVE-2023-2976 [hadoop-thirdparty]

2024-02-09 Thread via GitHub
steveloughran commented on PR #23: URL: https://github.com/apache/hadoop-thirdparty/pull/23#issuecomment-1936013649 new release is out; 3.4.0 RC2 will ship it! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Commented] (HADOOP-19059) S3A: update AWS SDK to 2.23.19 to support S3 Access Grants

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816079#comment-17816079 ] ASF GitHub Bot commented on HADOOP-19059: - steveloughran commented on PR #6506: URL:

Re: [PR] HADOOP-19059. Update AWS SDK to v2.23.7 [hadoop]

2024-02-09 Thread via GitHub
steveloughran commented on PR #6506: URL: https://github.com/apache/hadoop/pull/6506#issuecomment-1936011003 this has been obsoleted by #6538 which puts it up to 2.23.19, though it looks like there are more to come. Can I close it? -- This is an automated message from the Apache Git

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816078#comment-17816078 ] ASF GitHub Bot commented on HADOOP-19050: - steveloughran commented on PR #6507: URL:

Re: [PR] HADOOP-19050, Add Support for AWS S3 Access Grants [hadoop]

2024-02-09 Thread via GitHub
steveloughran commented on PR #6507: URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1936006860 @adnanhemani thanks; without that change we'd have problems with the PR, as in "you get to support it all through reflection" the way we have to do with wildfly/openssl binding

[jira] [Commented] (HADOOP-19057) S3 public test bucket landsat-pds unreadable -needs replacement

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816054#comment-17816054 ] ASF GitHub Bot commented on HADOOP-19057: - steveloughran commented on PR #6515: URL:

Re: [PR] HADOOP-19057. S3A: Landsat bucket used in tests no longer accessible [hadoop]

2024-02-09 Thread via GitHub
steveloughran commented on PR #6515: URL: https://github.com/apache/hadoop/pull/6515#issuecomment-1935903530 need urgent reviews/tests of this from anyone who can, just to fix the widespread test failures @ahmarsuhail @mukund-thakur @HarshitGupta11 @virajjasani @sunchao -- This

[jira] [Commented] (HADOOP-19069) Use hadoop-thirdparty 1.2.0

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816053#comment-17816053 ] ASF GitHub Bot commented on HADOOP-19069: - steveloughran merged PR #6541: URL:

[jira] [Updated] (HADOOP-19069) Use hadoop-thirdparty 1.2.0

2024-02-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19069: Fix Version/s: 3.3.9 > Use hadoop-thirdparty 1.2.0 > --- > >

Re: [PR] HADOOP-19069. Use hadoop-thirdparty 1.2.0. (#6533) [hadoop]

2024-02-09 Thread via GitHub
steveloughran merged PR #6541: URL: https://github.com/apache/hadoop/pull/6541 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-19069) Use hadoop-thirdparty 1.2.0

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816052#comment-17816052 ] ASF GitHub Bot commented on HADOOP-19069: - steveloughran commented on PR #6541: URL:

Re: [PR] HADOOP-19069. Use hadoop-thirdparty 1.2.0. (#6533) [hadoop]

2024-02-09 Thread via GitHub
steveloughran commented on PR #6541: URL: https://github.com/apache/hadoop/pull/6541#issuecomment-1935898164 reviewing tests, only hadoop-common tests were executed, no failures. going to +1 this. -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (HADOOP-19069) Use hadoop-thirdparty 1.2.0

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816051#comment-17816051 ] ASF GitHub Bot commented on HADOOP-19069: - steveloughran commented on PR #6541: URL:

Re: [PR] HADOOP-19069. Use hadoop-thirdparty 1.2.0. (#6533) [hadoop]

2024-02-09 Thread via GitHub
steveloughran commented on PR #6541: URL: https://github.com/apache/hadoop/pull/6541#issuecomment-1935893458 Merging this; all the warnings are just warnings of a future -in generated code, pretty meaningless -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (HADOOP-19069) Use hadoop-thirdparty 1.2.0

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816050#comment-17816050 ] ASF GitHub Bot commented on HADOOP-19069: - steveloughran commented on PR #6541: URL:

Re: [PR] HADOOP-19069. Use hadoop-thirdparty 1.2.0. (#6533) [hadoop]

2024-02-09 Thread via GitHub
steveloughran commented on PR #6541: URL: https://github.com/apache/hadoop/pull/6541#issuecomment-1935888966 There are lot of deprecation warnings here; mostly about PARSER ```

[jira] [Commented] (HADOOP-18679) Add API for bulk/paged object deletion

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816046#comment-17816046 ] ASF GitHub Bot commented on HADOOP-18679: - steveloughran commented on PR #6494: URL:

Re: [PR] HADOOP-18679. Add API for bulk/paged object deletion [hadoop]

2024-02-09 Thread via GitHub
steveloughran commented on PR #6494: URL: https://github.com/apache/hadoop/pull/6494#issuecomment-1935875347 +add a FileUtils method to assist deletion here, with `FileUtils.bulkDeletePageSize(path) -> int` and `FileUtils.bulkDelete(path, List) -> List; each will create a bulk delete

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816031#comment-17816031 ] ASF GitHub Bot commented on HADOOP-19050: - ahmarsuhail commented on code in PR #6544: URL:

Re: [PR] HADOOP-19050. Add S3 Access Grants Support in S3A [hadoop]

2024-02-09 Thread via GitHub
ahmarsuhail commented on code in PR #6544: URL: https://github.com/apache/hadoop/pull/6544#discussion_r1484099683 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.java: ## @@ -401,4 +411,19 @@ private static Region

Re: [PR] HDFS-17376. Distcp creates Factor 1 replication file on target if Source is EC. [hadoop]

2024-02-09 Thread via GitHub
sodonnel commented on PR #6540: URL: https://github.com/apache/hadoop/pull/6540#issuecomment-1935695635 @sadanand48 The change LGTM, but there are two checkstyle warnings in the test changes - could you fix them please? -- This is an automated message from the Apache Git Service. To

[jira] [Commented] (HADOOP-18342) Upgrade to Avro 1.11.1

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815998#comment-17815998 ] ASF GitHub Bot commented on HADOOP-18342: - hadoop-yetus commented on PR #4854: URL:

Re: [PR] HADOOP-18342: shaded avro jar [hadoop]

2024-02-09 Thread via GitHub
hadoop-yetus commented on PR #4854: URL: https://github.com/apache/hadoop/pull/4854#issuecomment-1935665039 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] [NOT FOR MERGE] test with shaded protobuf-java 3.21 (snapshot) [hadoop]

2024-02-09 Thread via GitHub
pjfanning closed pull request #6350: [NOT FOR MERGE] test with shaded protobuf-java 3.21 (snapshot) URL: https://github.com/apache/hadoop/pull/6350 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815984#comment-17815984 ] ASF GitHub Bot commented on HADOOP-19050: - hadoop-yetus commented on PR #6544: URL:

Re: [PR] HADOOP-19050. Add S3 Access Grants Support in S3A [hadoop]

2024-02-09 Thread via GitHub
hadoop-yetus commented on PR #6544: URL: https://github.com/apache/hadoop/pull/6544#issuecomment-1935575917 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |