[GitHub] [hadoop] hadoop-yetus commented on pull request #5469: HDFS-16943. RBF: Implements MySQL based StateStoreDriver.

2023-03-31 Thread via GitHub
hadoop-yetus commented on PR #5469: URL: https://github.com/apache/hadoop/pull/5469#issuecomment-1492818115 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] simbadzina commented on a diff in pull request #5469: HDFS-16943. RBF: Implements MySQL based StateStoreDriver.

2023-03-31 Thread via GitHub
simbadzina commented on code in PR #5469: URL: https://github.com/apache/hadoop/pull/5469#discussion_r1155038678 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/StateStoreDriver.java: ## @@ -35,6 +36,7 @@ * provider. D

[GitHub] [hadoop] simbadzina commented on a diff in pull request #5469: HDFS-16943. RBF: Implements MySQL based StateStoreDriver.

2023-03-31 Thread via GitHub
simbadzina commented on code in PR #5469: URL: https://github.com/apache/hadoop/pull/5469#discussion_r1155038043 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/impl/StateStoreMySQLImpl.java: ## @@ -0,0 +1,426 @@ +/** +

[GitHub] [hadoop] hadoop-yetus commented on pull request #5523: HDFS-16967. RBF: File based state stores should allow concurrent access to the records

2023-03-31 Thread via GitHub
hadoop-yetus commented on PR #5523: URL: https://github.com/apache/hadoop/pull/5523#issuecomment-1492784272 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] goiri commented on pull request #5495: HDFS-16949 Introduce inverse quantiles for metrics where higher numer…

2023-03-31 Thread via GitHub
goiri commented on PR #5495: URL: https://github.com/apache/hadoop/pull/5495#issuecomment-1492771329 Can we also get a clean build? I'm not sure if: * hadoop.hdfs.server.datanode.TestDirectoryScanner * hadoop.hdfs.server.namenode.ha.TestObserverNode * hadoop.hdfs.TestRollingUpgrad

[GitHub] [hadoop] goiri commented on pull request #5495: HDFS-16949 Introduce inverse quantiles for metrics where higher numer…

2023-03-31 Thread via GitHub
goiri commented on PR #5495: URL: https://github.com/apache/hadoop/pull/5495#issuecomment-1492770921 > The failed UTs seem flaky and unrelated to the change in this PR. They pass locally. @goiri Can you please help merging the PR? As he had comments, I'd like to get an approval from @

[GitHub] [hadoop] virajjasani commented on a diff in pull request #5523: HDFS-16967. RBF: File based state stores should allow concurrent access to the records

2023-03-31 Thread via GitHub
virajjasani commented on code in PR #5523: URL: https://github.com/apache/hadoop/pull/5523#discussion_r1155002213 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/impl/StateStoreFileBaseImpl.java: ## @@ -168,9 +182,30 @@

[GitHub] [hadoop] slfan1989 commented on pull request #5420: YARN-11442. Refactor FederationInterceptorREST Code.

2023-03-31 Thread via GitHub
slfan1989 commented on PR #5420: URL: https://github.com/apache/hadoop/pull/5420#issuecomment-1492724109 @goiri Thank you very much for helping to review the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [hadoop] goiri merged pull request #5420: YARN-11442. Refactor FederationInterceptorREST Code.

2023-03-31 Thread via GitHub
goiri merged PR #5420: URL: https://github.com/apache/hadoop/pull/5420 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.

[GitHub] [hadoop] goiri commented on a diff in pull request #5469: HDFS-16943. RBF: Implements MySQL based StateStoreDriver.

2023-03-31 Thread via GitHub
goiri commented on code in PR #5469: URL: https://github.com/apache/hadoop/pull/5469#discussion_r1154953596 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/StateStoreDriver.java: ## @@ -35,6 +36,7 @@ * provider. Driver

[GitHub] [hadoop] goiri commented on a diff in pull request #5523: HDFS-16967. RBF: File based state stores should allow concurrent access to the records

2023-03-31 Thread via GitHub
goiri commented on code in PR #5523: URL: https://github.com/apache/hadoop/pull/5523#discussion_r1154947181 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/impl/StateStoreFileBaseImpl.java: ## @@ -168,9 +182,30 @@ public

[GitHub] [hadoop] hadoop-yetus commented on pull request #5469: HDFS-16943. RBF: Implements MySQL based StateStoreDriver.

2023-03-31 Thread via GitHub
hadoop-yetus commented on PR #5469: URL: https://github.com/apache/hadoop/pull/5469#issuecomment-1492515280 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18012) ABFS: Enable config controlled ETag check for Rename idempotency

2023-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707388#comment-17707388 ] ASF GitHub Bot commented on HADOOP-18012: - steveloughran merged PR #5488: URL:

[GitHub] [hadoop] steveloughran merged pull request #5488: HADOOP-18012. ABFS: Using Source Path eTags for Rename Idemptonency checks

2023-03-31 Thread via GitHub
steveloughran merged PR #5488: URL: https://github.com/apache/hadoop/pull/5488 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...

[GitHub] [hadoop] hadoop-yetus commented on pull request #5519: MAPREDUCE-7435. Manifest Committer OOM on abfs

2023-03-31 Thread via GitHub
hadoop-yetus commented on PR #5519: URL: https://github.com/apache/hadoop/pull/5519#issuecomment-1492185475 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5520: HDFS-16965. Add switch to decide whether to enable native codec.

2023-03-31 Thread via GitHub
hadoop-yetus commented on PR #5520: URL: https://github.com/apache/hadoop/pull/5520#issuecomment-1492177793 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Created] (HADOOP-18688) s3a audit info to include #of items in a DeleteObjects request

2023-03-31 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18688: --- Summary: s3a audit info to include #of items in a DeleteObjects request Key: HADOOP-18688 URL: https://issues.apache.org/jira/browse/HADOOP-18688 Project: Hadoo

[jira] [Commented] (HADOOP-18012) ABFS: Enable config controlled ETag check for Rename idempotency

2023-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707308#comment-17707308 ] ASF GitHub Bot commented on HADOOP-18012: - hadoop-yetus commented on PR #5488:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5488: HADOOP-18012. ABFS: Using Source Path eTags for Rename Idemptonency checks

2023-03-31 Thread via GitHub
hadoop-yetus commented on PR #5488: URL: https://github.com/apache/hadoop/pull/5488#issuecomment-1492145022 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18012) ABFS: Enable config controlled ETag check for Rename idempotency

2023-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707306#comment-17707306 ] ASF GitHub Bot commented on HADOOP-18012: - hadoop-yetus commented on PR #5488:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5488: HADOOP-18012. ABFS: Using Source Path eTags for Rename Idemptonency checks

2023-03-31 Thread via GitHub
hadoop-yetus commented on PR #5488: URL: https://github.com/apache/hadoop/pull/5488#issuecomment-1492132562 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18685) Amazon S3 disabling ACLs on all new buckets

2023-03-31 Thread Daniel Carl Jones (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707295#comment-17707295 ] Daniel Carl Jones commented on HADOOP-18685: # When it fails, I believe we

[GitHub] [hadoop] steveloughran commented on pull request #5488: HADOOP-18012. ABFS: Using Source Path eTags for Rename Idemptonency checks

2023-03-31 Thread via GitHub
steveloughran commented on PR #5488: URL: https://github.com/apache/hadoop/pull/5488#issuecomment-1492060742 all good. once yetus gives the goahead i will merge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[jira] [Commented] (HADOOP-18012) ABFS: Enable config controlled ETag check for Rename idempotency

2023-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707289#comment-17707289 ] ASF GitHub Bot commented on HADOOP-18012: - steveloughran commented on PR #5488:

[jira] [Updated] (HADOOP-18012) ABFS: Enable config controlled ETag check for Rename idempotency

2023-03-31 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18012: Description: To support recovery of comms failure during rename, the abfs client fetches

[jira] [Commented] (HADOOP-18687) Remove unnecessary dependency on json-smart

2023-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707283#comment-17707283 ] ASF GitHub Bot commented on HADOOP-18687: - hadoop-yetus commented on PR #5524:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5524: HADOOP-18687 Remove unnecessary dependency on json-smart

2023-03-31 Thread via GitHub
hadoop-yetus commented on PR #5524: URL: https://github.com/apache/hadoop/pull/5524#issuecomment-1492018668 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Resolved] (HADOOP-18425) [ABFS]: RenameFilePath Source File Not Found (404) error in retry loop

2023-03-31 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18425. - Resolution: Duplicate > [ABFS]: RenameFilePath Source File Not Found (404) error in ret

[GitHub] [hadoop] hadoop-yetus commented on pull request #5521: HADOOP-18684: Fix S3A filesystem such that the scheme matches the URI scheme

2023-03-31 Thread via GitHub
hadoop-yetus commented on PR #5521: URL: https://github.com/apache/hadoop/pull/5521#issuecomment-1492007371 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18684) Fix S3A filesystem such that the scheme matches the URI scheme

2023-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707282#comment-17707282 ] ASF GitHub Bot commented on HADOOP-18684: - hadoop-yetus commented on PR #5521:

[jira] [Commented] (HADOOP-18685) Amazon S3 disabling ACLs on all new buckets

2023-03-31 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707280#comment-17707280 ] Steve Loughran commented on HADOOP-18685: - 1. what happens if the tests fail?

[jira] [Resolved] (HADOOP-18686) Backport of HADOOP-18493: Update jackson-databind for CVE fix for CVE-2022-42003

2023-03-31 Thread Diksha (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diksha resolved HADOOP-18686. - Resolution: Invalid > Backport of HADOOP-18493: Update jackson-databind for CVE fix for > CVE-2022-420

[jira] [Commented] (HADOOP-18012) ABFS: Enable config controlled ETag check for Rename idempotency

2023-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707261#comment-17707261 ] ASF GitHub Bot commented on HADOOP-18012: - sreeb-msft commented on code in PR #

[GitHub] [hadoop] sreeb-msft commented on a diff in pull request #5488: HADOOP-18012. ABFS: Using Source Path eTags for Rename Idemptonency checks

2023-03-31 Thread via GitHub
sreeb-msft commented on code in PR #5488: URL: https://github.com/apache/hadoop/pull/5488#discussion_r1154493316 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -79,6 +81,7 @@ import static org.apache.hadoop.fs.azurebfs.c

[jira] [Commented] (HADOOP-18012) ABFS: Enable config controlled ETag check for Rename idempotency

2023-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707247#comment-17707247 ] ASF GitHub Bot commented on HADOOP-18012: - steveloughran commented on code in P

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5488: HADOOP-18012. ABFS: Using Source Path eTags for Rename Idemptonency checks

2023-03-31 Thread via GitHub
steveloughran commented on code in PR #5488: URL: https://github.com/apache/hadoop/pull/5488#discussion_r1154411555 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -79,6 +81,7 @@ import static org.apache.hadoop.fs.azurebf

[jira] [Commented] (HADOOP-18684) Fix S3A filesystem such that the scheme matches the URI scheme

2023-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707246#comment-17707246 ] ASF GitHub Bot commented on HADOOP-18684: - steveloughran commented on code in P

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5521: HADOOP-18684: Fix S3A filesystem such that the scheme matches the URI scheme

2023-03-31 Thread via GitHub
steveloughran commented on code in PR #5521: URL: https://github.com/apache/hadoop/pull/5521#discussion_r1154393808 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AUrlScheme.java: ## @@ -1,31 +1,47 @@ +/* + * Licensed to the Apache Software Foundation (

[jira] [Commented] (HADOOP-18687) Remove unnecessary dependency on json-smart

2023-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707239#comment-17707239 ] ASF GitHub Bot commented on HADOOP-18687: - mjwiq commented on PR #5524: URL: ht

[GitHub] [hadoop] mjwiq commented on pull request #5524: HADOOP-18687 Remove unnecessary dependency on json-smart

2023-03-31 Thread via GitHub
mjwiq commented on PR #5524: URL: https://github.com/apache/hadoop/pull/5524#issuecomment-1491799598 I did not remove the entry from pom.xml in hadoop-project because it also sets the version for json-smart that is included in hdfs: ```[INFO] org.apache.hadoop:hadoop-hdfs-client:jar:3.4.

[jira] [Updated] (HADOOP-18687) Remove unnecessary dependency on json-smart

2023-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18687: Labels: pull-request-available (was: ) > Remove unnecessary dependency on json-smart > -

[jira] [Commented] (HADOOP-18687) Remove unnecessary dependency on json-smart

2023-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707237#comment-17707237 ] ASF GitHub Bot commented on HADOOP-18687: - mjwiq opened a new pull request, #55

[GitHub] [hadoop] mjwiq opened a new pull request, #5524: HADOOP-18687 Remove unnecessary dependency on json-smart

2023-03-31 Thread via GitHub
mjwiq opened a new pull request, #5524: URL: https://github.com/apache/hadoop/pull/5524 ### Description of PR https://issues.apache.org/jira/browse/HADOOP-18687 json-smart is not used by hadoop-auth and the reason for including it (for nimbus-jose-jwt) is no longer vali

[jira] [Created] (HADOOP-18687) Remove unnecessary dependency on json-smart

2023-03-31 Thread Michiel de Jong (Jira)
Michiel de Jong created HADOOP-18687: Summary: Remove unnecessary dependency on json-smart Key: HADOOP-18687 URL: https://issues.apache.org/jira/browse/HADOOP-18687 Project: Hadoop Common

[jira] [Commented] (HADOOP-18684) Fix S3A filesystem such that the scheme matches the URI scheme

2023-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707230#comment-17707230 ] ASF GitHub Bot commented on HADOOP-18684: - hadoop-yetus commented on PR #5521:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5521: HADOOP-18684: Fix S3A filesystem such that the scheme matches the URI scheme

2023-03-31 Thread via GitHub
hadoop-yetus commented on PR #5521: URL: https://github.com/apache/hadoop/pull/5521#issuecomment-1491697904 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18012) ABFS: Enable config controlled ETag check for Rename idempotency

2023-03-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707228#comment-17707228 ] ASF GitHub Bot commented on HADOOP-18012: - hadoop-yetus commented on PR #5488:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5488: HADOOP-18012. ABFS: Using Source Path eTags for Rename Idemptonency checks

2023-03-31 Thread via GitHub
hadoop-yetus commented on PR #5488: URL: https://github.com/apache/hadoop/pull/5488#issuecomment-1491679778 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Created] (HADOOP-18686) Backport of HADOOP-18493: Update jackson-databind for CVE fix for CVE-2022-42003

2023-03-31 Thread Diksha (Jira)
Diksha created HADOOP-18686: --- Summary: Backport of HADOOP-18493: Update jackson-databind for CVE fix for CVE-2022-42003 Key: HADOOP-18686 URL: https://issues.apache.org/jira/browse/HADOOP-18686 Project: Had