Re: [PR] HDFS-17397. Choose another DN as soon as possible, when encountering network issues [hadoop]

2024-02-26 Thread via GitHub
hadoop-yetus commented on PR #6591: URL: https://github.com/apache/hadoop/pull/6591#issuecomment-1965971951 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17387. [FGL] Abstract the configuration locking mode [hadoop]

2024-02-26 Thread via GitHub
ZanderXu commented on code in PR #6572: URL: https://github.com/apache/hadoop/pull/6572#discussion_r1503727005 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/fgl/FineGrainedFSNamesystemLock.java: ## Review Comment: Of course, I will

Re: [PR] HDFS-17387. [FGL] Abstract the configuration locking mode [hadoop]

2024-02-26 Thread via GitHub
ZanderXu commented on code in PR #6572: URL: https://github.com/apache/hadoop/pull/6572#discussion_r1503725124 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java: ## @@ -871,7 +876,10 @@ static FSNamesystem

[jira] [Commented] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820996#comment-17820996 ] ASF GitHub Bot commented on HADOOP-18516: - anmolanmol1234 commented on code in PR #6552: URL:

[jira] [Commented] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820997#comment-17820997 ] ASF GitHub Bot commented on HADOOP-18516: - anmolanmol1234 commented on code in PR #6552: URL:

Re: [PR] HADOOP-18516: [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication [hadoop]

2024-02-26 Thread via GitHub
anmolanmol1234 commented on code in PR #6552: URL: https://github.com/apache/hadoop/pull/6552#discussion_r1503722016 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/MockSASTokenProvider.java: ## @@ -32,8 +32,7 @@ import

Re: [PR] HADOOP-18516: [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication [hadoop]

2024-02-26 Thread via GitHub
anmolanmol1234 commented on code in PR #6552: URL: https://github.com/apache/hadoop/pull/6552#discussion_r1503721707 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemChooseSAS.java: ## @@ -128,11 +130,18 @@ public void

[jira] [Commented] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820995#comment-17820995 ] ASF GitHub Bot commented on HADOOP-18516: - anmolanmol1234 commented on code in PR #6552: URL:

Re: [PR] HADOOP-18516: [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication [hadoop]

2024-02-26 Thread via GitHub
anmolanmol1234 commented on code in PR #6552: URL: https://github.com/apache/hadoop/pull/6552#discussion_r1503719618 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemChooseSAS.java: ## @@ -97,16 +99,16 @@ public void

[jira] [Commented] (HADOOP-18910) ABFS: Adding Support for MD5 Hash based integrity verification of the request content during transport

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820993#comment-17820993 ] ASF GitHub Bot commented on HADOOP-18910: - anujmodi2021 commented on PR #6069: URL:

Re: [PR] HADOOP-18910: [ABFS] Adding Support for MD5 Hash based integrity verification of the request content during transport [hadoop]

2024-02-26 Thread via GitHub
anujmodi2021 commented on PR #6069: URL: https://github.com/apache/hadoop/pull/6069#issuecomment-1965889640 You also need to add following test configuration to specify the account type you are using ``fs.azure.test.namespace.enabled "`` ``` fs.azure.test.namespace.enabled

Re: [PR] HDFS-17387. [FGL] Abstract the configuration locking mode [hadoop]

2024-02-26 Thread via GitHub
ferhui commented on code in PR #6572: URL: https://github.com/apache/hadoop/pull/6572#discussion_r1503709489 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/fgl/FineGrainedFSNamesystemLock.java: ## Review Comment: How about

[jira] [Commented] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820990#comment-17820990 ] ASF GitHub Bot commented on HADOOP-18516: - anmolanmol1234 commented on code in PR #6552: URL:

Re: [PR] HADOOP-18516: [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication [hadoop]

2024-02-26 Thread via GitHub
anmolanmol1234 commented on code in PR #6552: URL: https://github.com/apache/hadoop/pull/6552#discussion_r1503713811 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/SASTokenProvider.java: ## @@ -51,13 +50,6 @@ public interface

[jira] [Commented] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820981#comment-17820981 ] ASF GitHub Bot commented on HADOOP-18516: - anujmodi2021 commented on code in PR #6552: URL:

Re: [PR] HADOOP-18516: [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication [hadoop]

2024-02-26 Thread via GitHub
anujmodi2021 commented on code in PR #6552: URL: https://github.com/apache/hadoop/pull/6552#discussion_r1503698211 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -332,6 +335,8 @@ public AbfsRestOperation

Re: [PR] HDFS-17387. [FGL] Abstract the configuration locking mode [hadoop]

2024-02-26 Thread via GitHub
ZanderXu commented on PR #6572: URL: https://github.com/apache/hadoop/pull/6572#issuecomment-1965866817 > Since [HDFS-17394](https://issues.apache.org/jira/browse/HDFS-17394) has been merged. can rebase this PR. done -- This is an automated message from the Apache Git Service. To

Re: [PR] HDFS-17387. [FGL] Abstract selectable locking mode [hadoop]

2024-02-26 Thread via GitHub
ferhui commented on PR #6572: URL: https://github.com/apache/hadoop/pull/6572#issuecomment-1965862096 Since HDFS-17394 has been merged. can rebase this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] HDFS-17394. [FGL] Remove unused WriteHoldCount of FSNamesystemLock [hadoop]

2024-02-26 Thread via GitHub
ferhui commented on PR #6571: URL: https://github.com/apache/hadoop/pull/6571#issuecomment-1965856467 @ZanderXu Thanks for this patch. @xinglin Thanks for reviewing it. Merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] HDFS-17394. [FGL] Remove unused WriteHoldCount of FSNamesystemLock [hadoop]

2024-02-26 Thread via GitHub
ferhui merged PR #6571: URL: https://github.com/apache/hadoop/pull/6571 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Updated] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation

2024-02-26 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-18516: --- Summary: [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider

[jira] [Updated] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to SAS Token Provider class

2024-02-26 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-18516: --- Fix Version/s: 3.4.0 > [ABFS]: Support fixed SAS token config in addition to SAS Token Provider

[jira] [Updated] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to SAS Token Provider class

2024-02-26 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-18516: --- Affects Version/s: 3.4.0 (was: 3.3.4) > [ABFS]: Support fixed SAS token

[jira] [Updated] (HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to SAS Token Provider class

2024-02-26 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-18516: --- Description: This PR introduces a new configuration for Fixed SAS Tokens: 

[jira] [Comment Edited] (HADOOP-19091) Add support for Tez to MagicS3GuardCommitter

2024-02-26 Thread Syed Shameerur Rahman (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820967#comment-17820967 ] Syed Shameerur Rahman edited comment on HADOOP-19091 at 2/27/24 5:35 AM:

[jira] [Commented] (HADOOP-19091) Add support for Tez to MagicS3GuardCommitter

2024-02-26 Thread Syed Shameerur Rahman (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820967#comment-17820967 ] Syed Shameerur Rahman commented on HADOOP-19091: [~vnarayanan7] - I am not sure why

[jira] [Commented] (HADOOP-19052) Hadoop use Shell command to get the count of the hard link which takes a lot of time

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820961#comment-17820961 ] ASF GitHub Bot commented on HADOOP-19052: - hadoop-yetus commented on PR #6587: URL:

Re: [PR] HADOOP-19052.Hadoop use Shell command to get the count of the hard link which takes a lot of time [hadoop]

2024-02-26 Thread via GitHub
hadoop-yetus commented on PR #6587: URL: https://github.com/apache/hadoop/pull/6587#issuecomment-1965800645 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Updated] (HADOOP-19089) [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path

2024-02-26 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-19089: --- Fix Version/s: 3.4.0 (was: 3.4.1) > [ABFS] Reverting Back Support of

[jira] [Updated] (HADOOP-19089) [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path

2024-02-26 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-19089: --- Affects Version/s: 3.4.0 > [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path

[jira] [Updated] (HADOOP-18869) ABFS: Fixing Behavior of a File System APIs on root path

2024-02-26 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-18869: --- Affects Version/s: 3.3.6 (was: 3.4.0) (was:

[jira] [Updated] (HADOOP-19089) [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path

2024-02-26 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-19089: --- Fix Version/s: 3.4.1 > [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path >

[jira] [Updated] (HADOOP-18869) ABFS: Fixing Behavior of a File System APIs on root path

2024-02-26 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-18869: --- Affects Version/s: 3.4.0 3.4.1 (was: 3.3.6) >

[jira] [Updated] (HADOOP-19089) [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path

2024-02-26 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-19089: --- Component/s: fs/azure > [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path >

Re: [PR] YARN-11626. Optimize ResourceManager's operations on Zookeeper metadata [hadoop]

2024-02-26 Thread via GitHub
hadoop-yetus commented on PR #6577: URL: https://github.com/apache/hadoop/pull/6577#issuecomment-1965764333 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17391:Adjust the checkpoint io buffer size to the chunk size [hadoop]

2024-02-26 Thread via GitHub
ThinkerLei closed pull request #6565: HDFS-17391:Adjust the checkpoint io buffer size to the chunk size URL: https://github.com/apache/hadoop/pull/6565 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] HDFS-17383:Datanode current block token should come from active NameNode in HA mode [hadoop]

2024-02-26 Thread via GitHub
ThinkerLei commented on code in PR #6562: URL: https://github.com/apache/hadoop/pull/6562#discussion_r1503615979 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java: ## @@ -2043,10 +2043,13 @@ public void

Re: [PR] HDFS-17383:Datanode current block token should come from active NameNode in HA mode [hadoop]

2024-02-26 Thread via GitHub
zhangshuyan0 commented on code in PR #6562: URL: https://github.com/apache/hadoop/pull/6562#discussion_r1503611470 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java: ## @@ -2043,10 +2043,13 @@ public void

Re: [PR] HDFS-17397. Choose another DN as soon as possible, when encountering network issues [hadoop]

2024-02-26 Thread via GitHub
hadoop-yetus commented on PR #6591: URL: https://github.com/apache/hadoop/pull/6591#issuecomment-1965754290 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17352. Add configuration to control whether DN delete this replica from disk when client requests a missing block [hadoop]

2024-02-26 Thread via GitHub
zhangshuyan0 commented on code in PR #6559: URL: https://github.com/apache/hadoop/pull/6559#discussion_r1503587650 ## hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml: ## @@ -3982,6 +3982,17 @@ + + dfs.datanode.delete.corrupt.replica.from.disk.enable

Re: [PR] HDFS-17396. BootstrapStandby should download rollback image during RollingUpgrade [hadoop]

2024-02-26 Thread via GitHub
hadoop-yetus commented on PR #6583: URL: https://github.com/apache/hadoop/pull/6583#issuecomment-1965705273 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17352. Add configuration to control whether DN delete this replica from disk when client requests a missing block [hadoop]

2024-02-26 Thread via GitHub
haiyang1987 commented on PR #6559: URL: https://github.com/apache/hadoop/pull/6559#issuecomment-1965674193 Hi @ZanderXu @tomscut @zhangshuyan0 @tasanuma please help me review again this PR when you are free, thanks ~ -- This is an automated message from the Apache Git Service. To

Re: [PR] HDFS-17358. EC: infinite lease recovery caused by the length of RWR equals to zero or datanode does not have the replica. [hadoop]

2024-02-26 Thread via GitHub
zhangshuyan0 commented on PR #6509: URL: https://github.com/apache/hadoop/pull/6509#issuecomment-1965670563 Committed to trunk. Thanks for your contributions! @hfutatzhanghb @haiyang1987 @tomscut -- This is an automated message from the Apache Git Service. To respond to the message,

[PR] HDFS-17391:Adjust the checkpoint io buffer size to the chunk size [hadoop]

2024-02-26 Thread via GitHub
ThinkerLei opened a new pull request, #6594: URL: https://github.com/apache/hadoop/pull/6594 Adjust the checkpoint io buffer size to the chunk size to reduce checkpoint time. Before change: 2022-07-11 07:10:50,900 INFO org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Uploaded

Re: [PR] HDFS-17358. EC: infinite lease recovery caused by the length of RWR equals to zero or datanode does not have the replica. [hadoop]

2024-02-26 Thread via GitHub
zhangshuyan0 merged PR #6509: URL: https://github.com/apache/hadoop/pull/6509 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HDFS-17365. EC: Add extra redunency configuration in checkStreamerFailures to prevent data loss. [hadoop]

2024-02-26 Thread via GitHub
hfutatzhanghb commented on code in PR #6517: URL: https://github.com/apache/hadoop/pull/6517#discussion_r1503518197 ## hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml: ## @@ -3908,6 +3908,18 @@ + +

[jira] [Commented] (HADOOP-19065) Update Protocol Buffers installation to 3.21.12

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820919#comment-17820919 ] ASF GitHub Bot commented on HADOOP-19065: - huangzhaobo99 commented on PR #6576: URL:

Re: [PR] HADOOP-19065. Update Protocol Buffers installation to 3.21.12 [hadoop]

2024-02-26 Thread via GitHub
huangzhaobo99 commented on PR #6576: URL: https://github.com/apache/hadoop/pull/6576#issuecomment-1965620933 > @huangzhaobo99 Thanks for the contribution! @steveloughran Thanks for the review! @slfan1989 Thanks for your verification and merging work, @steveloughran Thanks for your

Re: [PR] HDFS-17299. Adding rack failure tolerance when creating a new file [hadoop]

2024-02-26 Thread via GitHub
hadoop-yetus commented on PR #6566: URL: https://github.com/apache/hadoop/pull/6566#issuecomment-1965614194 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19091) Add support for Tez to MagicS3GuardCommitter

2024-02-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820885#comment-17820885 ] Steve Loughran commented on HADOOP-19091: - incidentally, look at {{buildJobUUID}} for different

[jira] [Commented] (HADOOP-18797) Support Concurrent Writes With S3A Magic Committer

2024-02-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820884#comment-17820884 ] Steve Loughran commented on HADOOP-18797: - [~srahman] FYI someone wants to support different

[jira] [Commented] (HADOOP-19091) Add support for Tez to MagicS3GuardCommitter

2024-02-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820882#comment-17820882 ] Steve Loughran commented on HADOOP-19091: - best place to start. be aware that the use of job id

[jira] [Updated] (HADOOP-19091) Add support for Tez to MagicS3GuardCommitter

2024-02-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19091: Affects Version/s: 3.3.6 3.4.0 (was:

[jira] [Updated] (HADOOP-19091) Add support for Tez to MagicS3GuardCommitter

2024-02-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19091: Issue Type: Improvement (was: Bug) > Add support for Tez to MagicS3GuardCommitter >

[jira] [Updated] (HADOOP-19091) Add support for Tez to MagicS3GuardCommitter

2024-02-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19091: Component/s: fs/s3 (was: tools) > Add support for Tez to

Re: [PR] HDFS-17396. BootstrapStandby should download rollback image during RollingUpgrade [hadoop]

2024-02-26 Thread via GitHub
hadoop-yetus commented on PR #6583: URL: https://github.com/apache/hadoop/pull/6583#issuecomment-1965318424 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17378. Add missing operationType for some operations in authorizer [hadoop]

2024-02-26 Thread via GitHub
hadoop-yetus commented on PR #6553: URL: https://github.com/apache/hadoop/pull/6553#issuecomment-1965290806 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18910) ABFS: Adding Support for MD5 Hash based integrity verification of the request content during transport

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820832#comment-17820832 ] ASF GitHub Bot commented on HADOOP-18910: - mukund-thakur commented on PR #6069: URL:

Re: [PR] HADOOP-18910: [ABFS] Adding Support for MD5 Hash based integrity verification of the request content during transport [hadoop]

2024-02-26 Thread via GitHub
mukund-thakur commented on PR #6069: URL: https://github.com/apache/hadoop/pull/6069#issuecomment-1965105331 ``` fs.azure.abfs.account.name account_name.dfs.core.windows.net fs.azure.account.auth.type.account_name.dfs.core.windows.net SharedKey

[jira] [Created] (HADOOP-19091) Add support for Tez to MagicS3GuardCommitter

2024-02-26 Thread Venkatasubrahmanian Narayanan (Jira)
Venkatasubrahmanian Narayanan created HADOOP-19091: -- Summary: Add support for Tez to MagicS3GuardCommitter Key: HADOOP-19091 URL: https://issues.apache.org/jira/browse/HADOOP-19091

[jira] [Commented] (HADOOP-19090) Update Protocol Buffers installation to 3.23.4

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820802#comment-17820802 ] ASF GitHub Bot commented on HADOOP-19090: - pjfanning opened a new pull request, #6593: URL:

[PR] HADOOP-19090. Use protobuf-java 3.23.4. [hadoop]

2024-02-26 Thread via GitHub
pjfanning opened a new pull request, #6593: URL: https://github.com/apache/hadoop/pull/6593 ### Description of PR DRAFT - See HADOOP-19090 The hadoop-thirdparty change is not merged yet so I am temporarily testing with a snapshot that I published with my groupId.

Re: [PR] HDFS-17358. EC: infinite lease recovery caused by the length of RWR equals to zero or datanode does not have replica. [hadoop]

2024-02-26 Thread via GitHub
hadoop-yetus commented on PR #6509: URL: https://github.com/apache/hadoop/pull/6509#issuecomment-1964732195 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17299. Adding rack failure tolerance when creating a new file [hadoop]

2024-02-26 Thread via GitHub
hadoop-yetus commented on PR #6566: URL: https://github.com/apache/hadoop/pull/6566#issuecomment-1964720571 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18708) AWS SDK V2 - Implement CSE

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820791#comment-17820791 ] ASF GitHub Bot commented on HADOOP-18708: - steveloughran commented on code in PR #6164: URL:

Re: [PR] HADOOP-18708. AWS SDK V2 - Implement CSE [hadoop]

2024-02-26 Thread via GitHub
steveloughran commented on code in PR #6164: URL: https://github.com/apache/hadoop/pull/6164#discussion_r1503019354 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/ErrorTranslation.java: ## @@ -152,6 +156,25 @@ public static IOException

[jira] [Updated] (HADOOP-19088) upgrade to jersey-json 1.22.0

2024-02-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19088: Component/s: build > upgrade to jersey-json 1.22.0 > - > >

[jira] [Updated] (HADOOP-19088) upgrade to jersey-json 1.22.0

2024-02-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19088: Affects Version/s: 3.3.6 > upgrade to jersey-json 1.22.0 > -

[jira] [Commented] (HADOOP-19088) upgrade to jersey-json 1.22.0

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820772#comment-17820772 ] ASF GitHub Bot commented on HADOOP-19088: - steveloughran commented on PR #6585: URL:

Re: [PR] HADOOP-19088. Use jersey-json 1.22.0 [hadoop]

2024-02-26 Thread via GitHub
steveloughran commented on PR #6585: URL: https://github.com/apache/hadoop/pull/6585#issuecomment-1964587995 not going to blame you for the javadocs; the unit test failures seem related to -protobuf update -maybe surefire update -- This is an automated message from the Apache Git

[jira] [Commented] (HADOOP-19089) [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path

2024-02-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820770#comment-17820770 ] Steve Loughran commented on HADOOP-19089: - can you tag with affected versions and module

[jira] [Updated] (HADOOP-19090) Update Protocol Buffers installation to 3.23.4

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-19090: Labels: pull-request-available (was: ) > Update Protocol Buffers installation to 3.23.4

[PR] HADOOP-19090. Use protobuf-java 3.23.4 [hadoop-thirdparty]

2024-02-26 Thread via GitHub
pjfanning opened a new pull request, #34: URL: https://github.com/apache/hadoop-thirdparty/pull/34 See HADOOP-19090 - this is still WIP and needs testing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Updated] (HADOOP-19084) prune dependency exports of hadoop-* modules

2024-02-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19084: Release Note: maven/ivy imports of hadoop-common are less likely to end up with log4j

[jira] [Updated] (HADOOP-19084) prune dependency exports of hadoop-* modules

2024-02-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19084: Fix Version/s: 3.5.0 3.4.1 > prune dependency exports of hadoop-*

[jira] [Commented] (HADOOP-19087) Release Hadoop 3.4.1

2024-02-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820767#comment-17820767 ] Steve Loughran commented on HADOOP-19087: - this is just the place to put stuff that we'd like

[jira] [Created] (HADOOP-19090) Update Protocol Buffers installation to 3.23.4

2024-02-26 Thread PJ Fanning (Jira)
PJ Fanning created HADOOP-19090: --- Summary: Update Protocol Buffers installation to 3.23.4 Key: HADOOP-19090 URL: https://issues.apache.org/jira/browse/HADOOP-19090 Project: Hadoop Common Issue

Re: [PR] HADOOP-19082: Update AWS SDK V2 to 2.24.1 [hadoop]

2024-02-26 Thread via GitHub
steveloughran commented on PR #6568: URL: https://github.com/apache/hadoop/pull/6568#issuecomment-1964533093 ok, the CSE stuff is going to fail, don't worry -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Commented] (HADOOP-14837) Handle S3A "glacier" data

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820759#comment-17820759 ] ASF GitHub Bot commented on HADOOP-14837: - steveloughran commented on PR #6407: URL:

Re: [PR] HADOOP-14837 : Support Read Restored Glacier Objects [hadoop]

2024-02-26 Thread via GitHub
steveloughran commented on PR #6407: URL: https://github.com/apache/hadoop/pull/6407#issuecomment-1964529624 I'm doing nothing but helping it get Hadoop 3.4.0 out the door this week. No code of my except related to packaging; no reviews of other people except related to the release at all.

Re: [PR] HDFS-17365. EC: Add extra redunency configuration in checkStreamerFailures to prevent data loss. [hadoop]

2024-02-26 Thread via GitHub
tasanuma commented on code in PR #6517: URL: https://github.com/apache/hadoop/pull/6517#discussion_r1502860434 ## hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml: ## @@ -3908,6 +3908,18 @@ + +

Re: [PR] YARN-11626. Optimize ResourceManager's operations on Zookeeper metadata [hadoop]

2024-02-26 Thread via GitHub
hadoop-yetus commented on PR #6577: URL: https://github.com/apache/hadoop/pull/6577#issuecomment-1964483913 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Updated] (HADOOP-19065) Update Protocol Buffers installation to 3.21.12

2024-02-26 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan updated HADOOP-19065: Fix Version/s: 3.4.1 > Update Protocol Buffers installation to 3.21.12 >

[jira] [Updated] (HADOOP-19065) Update Protocol Buffers installation to 3.21.12

2024-02-26 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan updated HADOOP-19065: Fix Version/s: 3.4.0 > Update Protocol Buffers installation to 3.21.12 >

[jira] [Commented] (HADOOP-19065) Update Protocol Buffers installation to 3.21.12

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820730#comment-17820730 ] ASF GitHub Bot commented on HADOOP-19065: - slfan1989 merged PR #6576: URL:

[jira] [Commented] (HADOOP-19065) Update Protocol Buffers installation to 3.21.12

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820731#comment-17820731 ] ASF GitHub Bot commented on HADOOP-19065: - slfan1989 commented on PR #6576: URL:

Re: [PR] HADOOP-19065. Update Protocol Buffers installation to 3.21.12 [hadoop]

2024-02-26 Thread via GitHub
slfan1989 commented on PR #6576: URL: https://github.com/apache/hadoop/pull/6576#issuecomment-1964331064 @huangzhaobo99 Thanks for the contribution! @steveloughran Thanks for the review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] HADOOP-19065. Update Protocol Buffers installation to 3.21.12 [hadoop]

2024-02-26 Thread via GitHub
slfan1989 merged PR #6576: URL: https://github.com/apache/hadoop/pull/6576 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569

2024-02-26 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820711#comment-17820711 ] Ayush Saxena commented on HADOOP-18197: --- If protobuf 3.23 has a fix & released, I think we should

[jira] [Commented] (HADOOP-19089) [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820710#comment-17820710 ] ASF GitHub Bot commented on HADOOP-19089: - hadoop-yetus commented on PR #6592: URL:

Re: [PR] HADOOP-19089: [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path [hadoop]

2024-02-26 Thread via GitHub
hadoop-yetus commented on PR #6592: URL: https://github.com/apache/hadoop/pull/6592#issuecomment-1964205785 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569

2024-02-26 Thread PJ Fanning (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820707#comment-17820707 ] PJ Fanning commented on HADOOP-18197: - The fix only seems to be in protobuf-java 3.23 and above -

Re: [PR] HDFS-17358. EC: infinite lease recovery caused by the length of RWR equals to zero or datanode does not have replica. [hadoop]

2024-02-26 Thread via GitHub
hadoop-yetus commented on PR #6509: URL: https://github.com/apache/hadoop/pull/6509#issuecomment-1964163458 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Updated] (HADOOP-18012) ABFS: Enable config controlled ETag check for Rename idempotency

2024-02-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18012: Description: To support recovery of comms failure during rename, the abfs client fetches

[jira] [Updated] (HADOOP-18012) ABFS: Enable config controlled ETag check for Rename idempotency

2024-02-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18012: Description: To support recovery of comms failure during rename, the abfs client fetches

Re: [PR] HDFS-17397. Choose another DN as soon as possible, when encountering network issues [hadoop]

2024-02-26 Thread via GitHub
hadoop-yetus commented on PR #6591: URL: https://github.com/apache/hadoop/pull/6591#issuecomment-1963984682 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19089) [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820681#comment-17820681 ] ASF GitHub Bot commented on HADOOP-19089: - saxenapranav commented on code in PR #6592: URL:

Re: [PR] HADOOP-19089: [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path [hadoop]

2024-02-26 Thread via GitHub
saxenapranav commented on code in PR #6592: URL: https://github.com/apache/hadoop/pull/6592#discussion_r1502450811 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemAttributes.java: ## @@ -91,13 +90,6 @@ public void

[jira] [Commented] (HADOOP-19089) [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820676#comment-17820676 ] ASF GitHub Bot commented on HADOOP-19089: - anujmodi2021 opened a new pull request, #6592: URL:

[jira] [Updated] (HADOOP-19089) [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path

2024-02-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-19089: Labels: pull-request-available (was: ) > [ABFS] Reverting Back Support of setXAttr()

  1   2   >