[GitHub] [hadoop] hadoop-yetus commented on pull request #5974: HDFS-17164. During rolling upgrade, datanode should copy-on-append when bumpReplicaGS.

2023-08-21 Thread via GitHub
hadoop-yetus commented on PR #5974: URL: https://github.com/apache/hadoop/pull/5974#issuecomment-1687345267 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] simbadzina commented on a diff in pull request #5971: YARN-11554. Fix TestRMFailover#testWebAppProxyInStandAloneMode Failed.

2023-08-21 Thread via GitHub
simbadzina commented on code in PR #5971: URL: https://github.com/apache/hadoop/pull/5971#discussion_r1300776343 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/TestRMFailover.java: ## @@ -212,6 +213,9 @@ public void

[GitHub] [hadoop] simbadzina commented on a diff in pull request #5971: YARN-11554. Fix TestRMFailover#testWebAppProxyInStandAloneMode Failed.

2023-08-21 Thread via GitHub
simbadzina commented on code in PR #5971: URL: https://github.com/apache/hadoop/pull/5971#discussion_r1300776343 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/TestRMFailover.java: ## @@ -212,6 +213,9 @@ public void

[jira] [Created] (HADOOP-18858) Upgrade guava due to CVE

2023-08-21 Thread Andrew Story (Jira)
Andrew Story created HADOOP-18858: - Summary: Upgrade guava due to CVE Key: HADOOP-18858 URL: https://issues.apache.org/jira/browse/HADOOP-18858 Project: Hadoop Common Issue Type: Bug

[jira] [Updated] (HADOOP-18858) Upgrade guava due to CVE

2023-08-21 Thread Andrew Story (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Story updated HADOOP-18858: -- Description: Update guava to 32.0.1 or higher due to CVE:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5974: HDFS-17164. During rolling upgrade, datanode should copy-on-append when bumpReplicaGS.

2023-08-21 Thread via GitHub
hadoop-yetus commented on PR #5974: URL: https://github.com/apache/hadoop/pull/5974#issuecomment-1687187357 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] szetszwo commented on a diff in pull request #5974: HDFS-17164. During rolling upgrade, datanode should copy-on-append when bumpReplicaGS.

2023-08-21 Thread via GitHub
szetszwo commented on code in PR #5974: URL: https://github.com/apache/hadoop/pull/5974#discussion_r1300668156 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/LocalReplica.java: ## @@ -367,12 +367,23 @@ public void

[GitHub] [hadoop] umamaheswararao commented on a diff in pull request #5974: HDFS-17164. During rolling upgrade, datanode should copy-on-append when bumpReplicaGS.

2023-08-21 Thread via GitHub
umamaheswararao commented on code in PR #5974: URL: https://github.com/apache/hadoop/pull/5974#discussion_r1300632520 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/LocalReplica.java: ## @@ -367,12 +367,23 @@ public void

[GitHub] [hadoop] umamaheswararao commented on a diff in pull request #5974: HDFS-17164. During rolling upgrade, datanode should copy-on-append when bumpReplicaGS.

2023-08-21 Thread via GitHub
umamaheswararao commented on code in PR #5974: URL: https://github.com/apache/hadoop/pull/5974#discussion_r1300559021 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/LocalReplica.java: ## @@ -367,12 +367,23 @@ public void

[GitHub] [hadoop] hadoop-yetus commented on pull request #5970: YARN-11535. Remove jackson-dataformat-yaml dependency.

2023-08-21 Thread via GitHub
hadoop-yetus commented on PR #5970: URL: https://github.com/apache/hadoop/pull/5970#issuecomment-1686840231 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5970: YARN-11535. Remove jackson-dataformat-yaml dependency.

2023-08-21 Thread via GitHub
hadoop-yetus commented on PR #5970: URL: https://github.com/apache/hadoop/pull/5970#issuecomment-1686816656 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5967: YARN-11435. [Router] FederationStateStoreFacade is not reinitialized with Router conf.

2023-08-21 Thread via GitHub
hadoop-yetus commented on PR #5967: URL: https://github.com/apache/hadoop/pull/5967#issuecomment-1686783181 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5972: Bump jquery from 3.3.1 to 3.5.0 in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yar

2023-08-21 Thread via GitHub
hadoop-yetus commented on PR #5972: URL: https://github.com/apache/hadoop/pull/5972#issuecomment-1686761503 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] szetszwo opened a new pull request, #5974: HDFS-17164. During rolling upgrade, datanode should copy-on-append when bumpReplicaGS.

2023-08-21 Thread via GitHub
szetszwo opened a new pull request, #5974: URL: https://github.com/apache/hadoop/pull/5974 ### Description of PR During rolling upgrade, datanode create a trash directory for each block pool. Then a block deletion becomes moving the block to trash. For bumpReplicaGS, it will

[jira] [Commented] (HADOOP-18820) AWS SDK v2: make the v1 bridging support optional

2023-08-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757014#comment-17757014 ] ASF GitHub Bot commented on HADOOP-18820: - steveloughran commented on PR #5973: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5973: HADOOP-18820. Cut AWS v1 support (#5872)

2023-08-21 Thread via GitHub
steveloughran commented on PR #5973: URL: https://github.com/apache/hadoop/pull/5973#issuecomment-1686682437 tested s3 london; mvit -Dparallel-tests -DtestsThreadCount=8 -Dscale existing multipart issue we should be able fix soon. if yetus is happy, i will merge this in ```

[GitHub] [hadoop] hadoop-yetus commented on pull request #5971: YARN-11554. Fix TestRMFailover#testWebAppProxyInStandAloneMode Failed.

2023-08-21 Thread via GitHub
hadoop-yetus commented on PR #5971: URL: https://github.com/apache/hadoop/pull/5971#issuecomment-1686628852 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] steveloughran opened a new pull request, #5973: HADOOP-18820. Cut AWS v1 support (#5872)

2023-08-21 Thread via GitHub
steveloughran opened a new pull request, #5973: URL: https://github.com/apache/hadoop/pull/5973 This removes the AWS V1 SDK as a hadoop-aws runtime dependency. It is still used at compile time so as to build a wrapper class V1ToV2AwsCredentialProviderAdapter which allows v1

[jira] [Commented] (HADOOP-18820) AWS SDK v2: make the v1 bridging support optional

2023-08-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756979#comment-17756979 ] ASF GitHub Bot commented on HADOOP-18820: - steveloughran opened a new pull request, #5973: URL:

[GitHub] [hadoop] brumi1024 merged pull request #5969: Revert "YARN-11535: Jackson-dataformat-yaml should be upgraded to 2.15.2 as it may cause transitive dependency issue with 2.12.7"

2023-08-21 Thread via GitHub
brumi1024 merged PR #5969: URL: https://github.com/apache/hadoop/pull/5969 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] brumi1024 commented on pull request #5969: Revert "YARN-11535: Jackson-dataformat-yaml should be upgraded to 2.15.2 as it may cause transitive dependency issue with 2.12.7"

2023-08-21 Thread via GitHub
brumi1024 commented on PR #5969: URL: https://github.com/apache/hadoop/pull/5969#issuecomment-1686576486 Thanks @susheelgupta7, merged to trunk. #5970 tracks the removal of jackson-dataformat-yaml. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop] dependabot[bot] opened a new pull request, #5972: Bump jquery from 3.3.1 to 3.5.0 in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop

2023-08-21 Thread via GitHub
dependabot[bot] opened a new pull request, #5972: URL: https://github.com/apache/hadoop/pull/5972 Bumps [jquery](https://github.com/jquery/jquery) from 3.3.1 to 3.5.0. Release notes Sourced from https://github.com/jquery/jquery/releases;>jquery's releases. jQuery 3.5.0

[GitHub] [hadoop] hadoop-yetus commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR

2023-08-21 Thread via GitHub
hadoop-yetus commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1686560867 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18820) AWS SDK v2: make the v1 bridging support optional

2023-08-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756973#comment-17756973 ] ASF GitHub Bot commented on HADOOP-18820: - steveloughran merged PR #5872: URL:

[GitHub] [hadoop] steveloughran merged pull request #5872: HADOOP-18820. Cut AWS v1 support

2023-08-21 Thread via GitHub
steveloughran merged PR #5872: URL: https://github.com/apache/hadoop/pull/5872 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18487) protobuf-2.5.0 dependencies => provided

2023-08-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756968#comment-17756968 ] ASF GitHub Bot commented on HADOOP-18487: - steveloughran commented on PR #4996: URL:

[GitHub] [hadoop] steveloughran commented on pull request #4996: HADOOP-18487. protobuf 2.5.0 marked as provided.

2023-08-21 Thread via GitHub
steveloughran commented on PR #4996: URL: https://github.com/apache/hadoop/pull/4996#issuecomment-1686526213 @pjfanning thanks -reviewed all the javadocs and corrected any issues I could see, made sure everything had a doc and used MUST NOT USE in the class javadoc. -- This is an

[jira] [Commented] (HADOOP-18487) protobuf-2.5.0 dependencies => provided

2023-08-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756966#comment-17756966 ] ASF GitHub Bot commented on HADOOP-18487: - pjfanning commented on code in PR #4996: URL:

[GitHub] [hadoop] pjfanning commented on a diff in pull request #4996: HADOOP-18487. protobuf 2.5.0 marked as provided.

2023-08-21 Thread via GitHub
pjfanning commented on code in PR #4996: URL: https://github.com/apache/hadoop/pull/4996#discussion_r1300238501 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufHelper.java: ## @@ -30,30 +30,35 @@ import

[GitHub] [hadoop-thirdparty] Killianoc commented on pull request #23: HADOOP-18843. Guava version 32.0.1 bump to fix CVE-2023-2976

2023-08-21 Thread via GitHub
Killianoc commented on PR #23: URL: https://github.com/apache/hadoop-thirdparty/pull/23#issuecomment-1686501911 Hi @jojochuang @fredbalves86, will this be available in a new release soon out of curiosity? -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569

2023-08-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756952#comment-17756952 ] ASF GitHub Bot commented on HADOOP-18197: - steveloughran commented on PR #4418: URL:

[GitHub] [hadoop] steveloughran commented on pull request #4418: HADOOP-18197. Upgrade protobuf to 3.21.x through upgraded hadoop-shaded-protobuf jar)

2023-08-21 Thread via GitHub
steveloughran commented on PR #4418: URL: https://github.com/apache/hadoop/pull/4418#issuecomment-1686463509 says 3.21.x... we should take the latest one we can which doesn't include other surprises...pr and jira can be set to the final version which goes in as it is merged -- This is

[jira] [Commented] (HADOOP-18496) upgrade kotlin-stdlib due to CVEs

2023-08-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756951#comment-17756951 ] ASF GitHub Bot commented on HADOOP-18496: - steveloughran commented on PR #5139: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5139: HADOOP-18496. Upgrade okhttp3 and dependencies due to kotlin CVEs

2023-08-21 Thread via GitHub
steveloughran commented on PR #5139: URL: https://github.com/apache/hadoop/pull/5139#issuecomment-1686459928 > I think that Hadoop has too many lib dependencies. I think hadoop-common has too many, and there are some which are particularly troublesome. we should really avoid adding

[jira] [Updated] (HADOOP-18856) Spark insertInto with location GCS bucket root not supported

2023-08-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18856: Summary: Spark insertInto with location GCS bucket root not supported (was: Spark

[GitHub] [hadoop] hadoop-yetus commented on pull request #5969: Revert "YARN-11535: Jackson-dataformat-yaml should be upgraded to 2.15.2 as it may cause transitive dependency issue with 2.12.7"

2023-08-21 Thread via GitHub
hadoop-yetus commented on PR #5969: URL: https://github.com/apache/hadoop/pull/5969#issuecomment-1686389895 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Updated] (HADOOP-18857) AWS v2 SDK: fail meaningfully if legacy v2 signing requested

2023-08-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18857: Description: if the v2 signer is requested on an AWS SDKv2 build, you get an NPE.

[jira] [Created] (HADOOP-18857) AWS v2 SDK: fail meaningfully if legacy v2 signing requested

2023-08-21 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18857: --- Summary: AWS v2 SDK: fail meaningfully if legacy v2 signing requested Key: HADOOP-18857 URL: https://issues.apache.org/jira/browse/HADOOP-18857 Project: Hadoop

[GitHub] [hadoop] slfan1989 opened a new pull request, #5971: YARN-11554. Fix TestRMFailover#testWebAppProxyInStandAloneMode Failed.

2023-08-21 Thread via GitHub
slfan1989 opened a new pull request, #5971: URL: https://github.com/apache/hadoop/pull/5971 ### Description of PR JIRA: YARN-11554. Fix TestRMFailover#testWebAppProxyInStandAloneMode Failed. ### How was this patch tested? ### For code changes: - [ ]

[GitHub] [hadoop] fanluoo commented on pull request #5591: HDFS-16991. fix testMkdirsRaceWithObserverRead

2023-08-21 Thread via GitHub
fanluoo commented on PR #5591: URL: https://github.com/apache/hadoop/pull/5591#issuecomment-1686300845 @ayushtkn now it`s green! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hadoop] brumi1024 opened a new pull request, #5970: YARN-11535. Remove jackson-dataformat-xml as a dependency.

2023-08-21 Thread via GitHub
brumi1024 opened a new pull request, #5970: URL: https://github.com/apache/hadoop/pull/5970 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g.

[GitHub] [hadoop] susheelgupta7 opened a new pull request, #5969: Revert "YARN-11535: Jackson-dataformat-yaml should be upgraded to 2.15.2 as it may cause transitive dependency issue with 2.12.7"

2023-08-21 Thread via GitHub
susheelgupta7 opened a new pull request, #5969: URL: https://github.com/apache/hadoop/pull/5969 Reverts apache/hadoop#5884 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hadoop] zhangxiping1 commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…

2023-08-21 Thread via GitHub
zhangxiping1 commented on PR #5921: URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1686158033 @goiri Modified according to your suggestion, can you help to see what is wrong with the test case? -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Comment Edited] (HADOOP-18856) Spark insertInto with location GCS bucket root causes NPE

2023-08-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756834#comment-17756834 ] Steve Loughran edited comment on HADOOP-18856 at 8/21/23 11:09 AM: ---

[jira] [Commented] (HADOOP-18856) Spark insertInto with location GCS bucket root causes NPE

2023-08-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756834#comment-17756834 ] Steve Loughran commented on HADOOP-18856: - bq. Dataproc's latest 2.1 uses Hadoop 3.3.3 and this

[GitHub] [hadoop] hadoop-yetus commented on pull request #5957: YARN-6537. Consider running RM tests against the Router.

2023-08-21 Thread via GitHub
hadoop-yetus commented on PR #5957: URL: https://github.com/apache/hadoop/pull/5957#issuecomment-1686053611 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5944: YARN-11537. [Federation] Router CLI Supports List SubClusterPolicyConfiguration Of Queues.

2023-08-21 Thread via GitHub
hadoop-yetus commented on PR #5944: URL: https://github.com/apache/hadoop/pull/5944#issuecomment-1685903278 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5967: YARN-11435. [Router] FederationStateStoreFacade is not reinitialized with Router conf.

2023-08-21 Thread via GitHub
hadoop-yetus commented on PR #5967: URL: https://github.com/apache/hadoop/pull/5967#issuecomment-1685902048 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5944: YARN-11537. [Federation] Router CLI Supports List SubClusterPolicyConfiguration Of Queues.

2023-08-21 Thread via GitHub
hadoop-yetus commented on PR #5944: URL: https://github.com/apache/hadoop/pull/5944#issuecomment-1685893079 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR

2023-08-21 Thread via GitHub
hadoop-yetus commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1685882446 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |