Re: [PR] [ABFS] [Backport 3.4] Back Merging PRs from trunk to Branch 3.4 [hadoop]

2024-03-04 Thread via GitHub
hadoop-yetus commented on PR #6611: URL: https://github.com/apache/hadoop/pull/6611#issuecomment-1978079289 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17396. BootstrapStandby should download rollback image during RollingUpgrade [hadoop]

2024-03-04 Thread via GitHub
hadoop-yetus commented on PR #6583: URL: https://github.com/apache/hadoop/pull/6583#issuecomment-1978026417 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HADOOP-18980. Invalid inputs for getTrimmedStringCollectionSplitByEquals (ADDENDUM) [hadoop]

2024-03-04 Thread via GitHub
virajjasani commented on PR #6546: URL: https://github.com/apache/hadoop/pull/6546#issuecomment-1978003122 @steveloughran a gentle reminder :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[PR] [ABFS] [Backport 3.4] Back Merging PRs from trunk to Branch 3.4 [hadoop]

2024-03-04 Thread via GitHub
anujmodi2021 opened a new pull request, #6611: URL: https://github.com/apache/hadoop/pull/6611 ### Description of PR Following two PRs were merged to trunk. #5881 #6069 These two PRs may have conflicts thus creating this single backport PR to avoid any merge issues.

Re: [PR] Test build [hadoop]

2024-03-04 Thread via GitHub
hadoop-yetus commented on PR #6610: URL: https://github.com/apache/hadoop/pull/6610#issuecomment-1977963439 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17398. [FGL] Implement the FGL lock for FSNLockManager [hadoop]

2024-03-04 Thread via GitHub
ferhui commented on code in PR #6599: URL: https://github.com/apache/hadoop/pull/6599#discussion_r1512099828 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/fgl/FineGrainedFSNamesystemLock.java: ## @@ -0,0 +1,293 @@ +/** + * Licensed to

Re: [PR] HDFS-17299. Adding rack failure tolerance when creating a new file [hadoop]

2024-03-04 Thread via GitHub
hadoop-yetus commented on PR #6566: URL: https://github.com/apache/hadoop/pull/6566#issuecomment-1977869072 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17404. Add NN Socket Address to log when processing command from active NN [hadoop]

2024-03-04 Thread via GitHub
hadoop-yetus commented on PR #6609: URL: https://github.com/apache/hadoop/pull/6609#issuecomment-1977694056 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[PR] Test build [hadoop]

2024-03-04 Thread via GitHub
ritegarg opened a new pull request, #6610: URL: https://github.com/apache/hadoop/pull/6610 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g.

Re: [PR] HDFS-17299. Adding rack failure tolerance when creating a new file [hadoop]

2024-03-04 Thread via GitHub
hadoop-yetus commented on PR #6566: URL: https://github.com/apache/hadoop/pull/6566#issuecomment-1977657875 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19098) Vector IO: consistent specified rejection of overlapping ranges

2024-03-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823376#comment-17823376 ] ASF GitHub Bot commented on HADOOP-19098: - hadoop-yetus commented on PR #6604: URL:

Re: [PR] HADOOP-19098 Vector IO: consistent specified rejection of overlapping ranges [hadoop]

2024-03-04 Thread via GitHub
hadoop-yetus commented on PR #6604: URL: https://github.com/apache/hadoop/pull/6604#issuecomment-1977653680 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17299. Adding rack failure tolerance when creating a new file [hadoop]

2024-03-04 Thread via GitHub
hadoop-yetus commented on PR #6566: URL: https://github.com/apache/hadoop/pull/6566#issuecomment-1977608476 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Updated] (HADOOP-19018) Release Hadoop 3.4.0

2024-03-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-19018: Labels: pull-request-available (was: ) > Release Hadoop 3.4.0 > >

[jira] [Commented] (HADOOP-18583) hadoop checknative fails to load openssl 3.x

2024-03-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823345#comment-17823345 ] ASF GitHub Bot commented on HADOOP-18583: - steveloughran commented on PR #5256: URL:

Re: [PR] HADOOP-18583. Fix loading of OpenSSL 3.x symbols [hadoop]

2024-03-04 Thread via GitHub
steveloughran commented on PR #5256: URL: https://github.com/apache/hadoop/pull/5256#issuecomment-1977458743 > hadoop-yarn-server-nodemanager's container-executor no idea. best to discuss on yarn-dev regarding crypton and checknative, we may to work on that. IMO checknative should

Re: [PR] HDFS-17299. Adding rack failure tolerance when creating a new file [hadoop]

2024-03-04 Thread via GitHub
ayushtkn commented on code in PR #6566: URL: https://github.com/apache/hadoop/pull/6566#discussion_r1511777291 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java: ## @@ -1817,10 +1839,10 @@ protected LocatedBlock

[jira] [Created] (HADOOP-19101) Vectored Read into off-heap buffer broken

2024-03-04 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19101: --- Summary: Vectored Read into off-heap buffer broken Key: HADOOP-19101 URL: https://issues.apache.org/jira/browse/HADOOP-19101 Project: Hadoop Common

[jira] [Updated] (HADOOP-18855) VectorIO API tuning

2024-03-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18855: Affects Version/s: 3.4.0 > VectorIO API tuning > --- > >

[jira] [Assigned] (HADOOP-18855) VectorIO API tuning

2024-03-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18855: --- Assignee: Steve Loughran > VectorIO API tuning > --- > >

Re: [PR] HDFS-17383:Datanode current block token should come from active NameNode in HA mode [hadoop]

2024-03-04 Thread via GitHub
hadoop-yetus commented on PR #6562: URL: https://github.com/apache/hadoop/pull/6562#issuecomment-1977312121 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19098) Vector IO: consistent specified rejection of overlapping ranges

2024-03-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823287#comment-17823287 ] ASF GitHub Bot commented on HADOOP-19098: - steveloughran commented on PR #6604: URL:

Re: [PR] HADOOP-19098 Vector IO: consistent specified rejection of overlapping ranges [hadoop]

2024-03-04 Thread via GitHub
steveloughran commented on PR #6604: URL: https://github.com/apache/hadoop/pull/6604#issuecomment-1977235222 Just going to highlight that the contract test failed *badly* when reading into direct buffers from azure. I think this is a bug in the direct buffer fetching logic -it's always

Re: [PR] HDFS-17408:Reduce the number of quota calculations in FSDirRenameOp [hadoop]

2024-03-04 Thread via GitHub
hadoop-yetus commented on PR #6608: URL: https://github.com/apache/hadoop/pull/6608#issuecomment-1977212481 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17398. [FGL] Implement the FGL lock for FSNLockManager [hadoop]

2024-03-04 Thread via GitHub
hadoop-yetus commented on PR #6599: URL: https://github.com/apache/hadoop/pull/6599#issuecomment-1977198415 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Comment Edited] (HADOOP-19091) Add support for Tez to MagicS3GuardCommitter

2024-03-04 Thread Venkatasubrahmanian Narayanan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823282#comment-17823282 ] Venkatasubrahmanian Narayanan edited comment on HADOOP-19091 at 3/4/24 6:23 PM:

[jira] [Commented] (HADOOP-19091) Add support for Tez to MagicS3GuardCommitter

2024-03-04 Thread Venkatasubrahmanian Narayanan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823282#comment-17823282 ] Venkatasubrahmanian Narayanan commented on HADOOP-19091: [~srahman] While

[jira] [Comment Edited] (HADOOP-19091) Add support for Tez to MagicS3GuardCommitter

2024-03-04 Thread Venkatasubrahmanian Narayanan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823282#comment-17823282 ] Venkatasubrahmanian Narayanan edited comment on HADOOP-19091 at 3/4/24 6:22 PM:

[PR] HDFS-17404: Add NN Socket Address to log when processing command from active NN [hadoop]

2024-03-04 Thread via GitHub
Jtdellaringa opened a new pull request, #6609: URL: https://github.com/apache/hadoop/pull/6609 ### Description of PR Add NN Socket Address to log when processing command from active NN ### How was this patch tested? n/a ### For code changes: - [ x] Does the title

Re: [PR] HDFS-17404. Add Namenode info to log message when setting block keys [hadoop]

2024-03-04 Thread via GitHub
Jtdellaringa commented on code in PR #6598: URL: https://github.com/apache/hadoop/pull/6598#discussion_r1511568416 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java: ## @@ -226,7 +228,7 @@ public

Re: [PR] HDFS-17299. Adding rack failure tolerance when creating a new file [hadoop]

2024-03-04 Thread via GitHub
shahrs87 commented on code in PR #6566: URL: https://github.com/apache/hadoop/pull/6566#discussion_r1511445056 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java: ## @@ -1817,10 +1839,10 @@ protected LocatedBlock

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-03-04 Thread via GitHub
hadoop-yetus commented on PR #6314: URL: https://github.com/apache/hadoop/pull/6314#issuecomment-1976952564 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17391:Adjust the checkpoint io buffer size to the chunk size [hadoop]

2024-03-04 Thread via GitHub
hadoop-yetus commented on PR #6594: URL: https://github.com/apache/hadoop/pull/6594#issuecomment-1976950545 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-03-04 Thread via GitHub
hadoop-yetus commented on PR #6314: URL: https://github.com/apache/hadoop/pull/6314#issuecomment-1976675970 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-03-04 Thread via GitHub
hadoop-yetus commented on PR #6314: URL: https://github.com/apache/hadoop/pull/6314#issuecomment-1976639659 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[PR] HDFS-17408:Reduce the number of quota calculations in FSDirRenameOp [hadoop]

2024-03-04 Thread via GitHub
ThinkerLei opened a new pull request, #6608: URL: https://github.com/apache/hadoop/pull/6608 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[jira] [Commented] (HADOOP-19097) core-default fs.s3a.connection.establish.timeout value too low -warning always printed

2024-03-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823153#comment-17823153 ] ASF GitHub Bot commented on HADOOP-19097: - steveloughran commented on PR #6601: URL:

Re: [PR] HADOOP-19097. S3A: Set fs.s3a.connection.establish.timeout to 30s [hadoop]

2024-03-04 Thread via GitHub
steveloughran commented on PR #6601: URL: https://github.com/apache/hadoop/pull/6601#issuecomment-1976473456 @virajjasani I did a scan. Now, the clever thing would be to have a test suite which compared values so there is never a regression. e.g assertDurationEqual(conf,

Re: [PR] HDFS-17387. [FGL] Abstract the configuration locking mode [hadoop]

2024-03-04 Thread via GitHub
ferhui commented on PR #6572: URL: https://github.com/apache/hadoop/pull/6572#issuecomment-1976195600 Thanks. Merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] HDFS-17387. [FGL] Abstract the configuration locking mode [hadoop]

2024-03-04 Thread via GitHub
ferhui merged PR #6572: URL: https://github.com/apache/hadoop/pull/6572 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HDFS-17387. [FGL] Abstract the configuration locking mode [hadoop]

2024-03-04 Thread via GitHub
ZanderXu commented on PR #6572: URL: https://github.com/apache/hadoop/pull/6572#issuecomment-1976061312 @ferhui The failed UTs are not introduced by this PR. I will submit a new PR to fix them and contribute them to the trunk branch. -- This is an automated message from the Apache Git

Re: [PR] HDFS-17387. [FGL] Abstract the configuration locking mode [hadoop]

2024-03-04 Thread via GitHub
hadoop-yetus commented on PR #6572: URL: https://github.com/apache/hadoop/pull/6572#issuecomment-1976013604 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19085) Compatibility Benchmark over HCFS Implementations

2024-03-04 Thread Kai Zheng (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823076#comment-17823076 ] Kai Zheng commented on HADOOP-19085: Hi [~han.liu] The work looks great overall. Some comments

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-03-04 Thread via GitHub
saxenapranav commented on code in PR #6314: URL: https://github.com/apache/hadoop/pull/6314#discussion_r1510743018 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1411,6 +1448,97 @@ protected AccessTokenProvider

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-03-04 Thread via GitHub
saxenapranav commented on code in PR #6314: URL: https://github.com/apache/hadoop/pull/6314#discussion_r1510743018 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1411,6 +1448,97 @@ protected AccessTokenProvider

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-03-04 Thread via GitHub
anmolanmol1234 commented on code in PR #6314: URL: https://github.com/apache/hadoop/pull/6314#discussion_r1510739331 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1411,6 +1448,97 @@ protected AccessTokenProvider