[GitHub] [hadoop] hadoop-yetus commented on pull request #5990: HDFS-17166. RBF: Throwing NoNamenodesAvailableException for a long time, when failover

2023-08-30 Thread via GitHub
hadoop-yetus commented on PR #5990: URL: https://github.com/apache/hadoop/pull/5990#issuecomment-1698987513 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] steveloughran commented on pull request #5995: HADOOP-18818. Merge aws v2 upgrade feature branch into trunk

2023-08-30 Thread via GitHub
steveloughran commented on PR #5995: URL: https://github.com/apache/hadoop/pull/5995#issuecomment-1699098925 right, i'm just going to comment out the enforcement, as a package wildcard didn't take either -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hadoop] hadoop-yetus commented on pull request #5993: HADOOP-18679. Add API for bulk/paged object deletion

2023-08-30 Thread via GitHub
hadoop-yetus commented on PR #5993: URL: https://github.com/apache/hadoop/pull/5993#issuecomment-1699114687 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] yangjiandan closed pull request #5978: YARN-11552. Timeline endpoint: /clusters/{clusterid}/apps/{appid}/entity-types Error when using hdfs store

2023-08-30 Thread via GitHub
yangjiandan closed pull request #5978: YARN-11552. Timeline endpoint: /clusters/{clusterid}/apps/{appid}/entity-types Error when using hdfs store URL: https://github.com/apache/hadoop/pull/5978 -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hadoop] hadoop-yetus commented on pull request #6000: HADOOP-18257. Merging and Parsing S3A audit logs into Avro format for analysis.

2023-08-30 Thread via GitHub
hadoop-yetus commented on PR #6000: URL: https://github.com/apache/hadoop/pull/6000#issuecomment-1698859376 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5978: YARN-11552. Timeline endpoint: /clusters/{clusterid}/apps/{appid}/entity-types Error when using hdfs store

2023-08-30 Thread via GitHub
hadoop-yetus commented on PR #5978: URL: https://github.com/apache/hadoop/pull/5978#issuecomment-1698980626 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR

2023-08-30 Thread via GitHub
hadoop-yetus commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1698736321 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5995: HADOOP-18818. Merge aws v2 upgrade feature branch into trunk

2023-08-30 Thread via GitHub
hadoop-yetus commented on PR #5995: URL: https://github.com/apache/hadoop/pull/5995#issuecomment-1698782448 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] haiyang1987 opened a new pull request, #6002: HDFS-17172. Support FSNamesystemLock Parameters reconfigurable

2023-08-30 Thread via GitHub
haiyang1987 opened a new pull request, #6002: URL: https://github.com/apache/hadoop/pull/6002 ### Description of PR https://issues.apache.org/jira/browse/HDFS-17172 For Namesystem lock some parameters such as "dfs.namenode.lock.detailed-metrics.enabled",

[GitHub] [hadoop] anujmodi2021 opened a new pull request, #6004: HADOOP-18874: [ABFS] Adding Server returned request id in Exception Message thrown to caller.

2023-08-30 Thread via GitHub
anujmodi2021 opened a new pull request, #6004: URL: https://github.com/apache/hadoop/pull/6004 Jira Ticket: https://issues.apache.org/jira/browse/HADOOP-18874 ### Description of PR Each request made to Azure server has its unique ActivityId (rid) which is returned in response of

[GitHub] [hadoop] hadoop-yetus commented on pull request #5990: HDFS-17166. RBF: Throwing NoNamenodesAvailableException for a long time, when failover

2023-08-30 Thread via GitHub
hadoop-yetus commented on PR #5990: URL: https://github.com/apache/hadoop/pull/5990#issuecomment-1698769109 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5881: Hadoop-18759: [ABFS][Backoff-Optimization] Have a Static retry policy for connection timeout.

2023-08-30 Thread via GitHub
hadoop-yetus commented on PR #5881: URL: https://github.com/apache/hadoop/pull/5881#issuecomment-1698838047 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] vikaskr22 opened a new pull request, #6001: HADOOP-18851: Perfm improvement for ZKDT management

2023-08-30 Thread via GitHub
vikaskr22 opened a new pull request, #6001: URL: https://github.com/apache/hadoop/pull/6001 ### Description of PR AbstractDelegationTokenSecretManager's method all synchronized and are blocking each other, even multiple readers threads are blocking each other. This PR is an

[GitHub] [hadoop] yangjiandan commented on pull request #5978: YARN-11552. Timeline endpoint: /clusters/{clusterid}/apps/{appid}/entity-types Error when using hdfs store

2023-08-30 Thread via GitHub
yangjiandan commented on PR #5978: URL: https://github.com/apache/hadoop/pull/5978#issuecomment-1698987308 @slfan1989 have you ever encountered this shadedclient error? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] anujmodi2021 opened a new pull request, #6003: HADOOP-18869: [ABFS] Fixing Behavior of a File System APIs on root path

2023-08-30 Thread via GitHub
anujmodi2021 opened a new pull request, #6003: URL: https://github.com/apache/hadoop/pull/6003 ### Description of PR Jira Ticket: https://issues.apache.org/jira/browse/HADOOP-18869 Following changes are included in this PR: 1. HDFS.create() was failing on root path with Invalid

[GitHub] [hadoop] vikaskr22 commented on pull request #6001: HADOOP-18851: Perfm improvement for ZKDT management

2023-08-30 Thread via GitHub
vikaskr22 commented on PR #6001: URL: https://github.com/apache/hadoop/pull/6001#issuecomment-1698903666 @jojochuang , as per our discussion on JIRA, this is the PR. Please review and provide your inputs. Thanks. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hadoop] KeeProMise commented on pull request #5990: HDFS-17166. RBF: Throwing NoNamenodesAvailableException for a long time, when failover

2023-08-30 Thread via GitHub
KeeProMise commented on PR #5990: URL: https://github.com/apache/hadoop/pull/5990#issuecomment-1699004525 I think the shadedclient error is not related to my pr, because I found that other PRs in the recent period also have this problem. -- This is an automated message from the Apache

[GitHub] [hadoop] vikaskr22 commented on a diff in pull request #6001: HADOOP-18851: Performance improvement for DelegationTokenSecretManager.

2023-09-05 Thread via GitHub
vikaskr22 commented on code in PR #6001: URL: https://github.com/apache/hadoop/pull/6001#discussion_r1315712962 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java: ## @@ -520,24 +525,32 @@

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

2023-09-05 Thread via GitHub
slfan1989 commented on code in PR #6016: URL: https://github.com/apache/hadoop/pull/6016#discussion_r1315733198 ##

[GitHub] [hadoop] hadoop-yetus commented on pull request #6012: YARN-11433. Router's main() should support generic options.

2023-09-05 Thread via GitHub
hadoop-yetus commented on PR #6012: URL: https://github.com/apache/hadoop/pull/6012#issuecomment-1706504451 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5920: YARN-6476. Advanced Federation UI based on YARN UI v2.

2023-09-05 Thread via GitHub
hadoop-yetus commented on PR #5920: URL: https://github.com/apache/hadoop/pull/5920#issuecomment-1706504765 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #6022: Expect100resolution

2023-09-05 Thread via GitHub
hadoop-yetus commented on PR #6022: URL: https://github.com/apache/hadoop/pull/6022#issuecomment-1706546723 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

2023-09-05 Thread via GitHub
hadoop-yetus commented on PR #6016: URL: https://github.com/apache/hadoop/pull/6016#issuecomment-1706593281 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

2023-09-05 Thread via GitHub
hadoop-yetus commented on PR #6016: URL: https://github.com/apache/hadoop/pull/6016#issuecomment-1706654372 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 opened a new pull request, #6023: YARN-11562. [Federation] GPG Support Query Policies In Web.

2023-09-05 Thread via GitHub
slfan1989 opened a new pull request, #6023: URL: https://github.com/apache/hadoop/pull/6023 ### Description of PR JIRA: YARN-11562. [Federation] GPG Support Query Policies In Web. GPG should provide the functionality to query queue policies through the web interface.

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

2023-09-05 Thread via GitHub
slfan1989 commented on code in PR #6016: URL: https://github.com/apache/hadoop/pull/6016#discussion_r1315732505 ##

[GitHub] [hadoop] hadoop-yetus commented on pull request #6011: YARN-11434. [Router] UGI conf doesn't read user overridden configurations on Router startup.

2023-09-05 Thread via GitHub
hadoop-yetus commented on PR #6011: URL: https://github.com/apache/hadoop/pull/6011#issuecomment-1706515651 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #6007: HADOOP-18870. CURATOR-599 change broke functionality introduced in HA…

2023-09-05 Thread via GitHub
hadoop-yetus commented on PR #6007: URL: https://github.com/apache/hadoop/pull/6007#issuecomment-1706772742 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on pull request #6012: YARN-11433. Router's main() should support generic options.

2023-09-05 Thread via GitHub
slfan1989 commented on PR #6012: URL: https://github.com/apache/hadoop/pull/6012#issuecomment-1706771719 @goiri Can you help review this PR again? Thank you very much! I read the code of `RM` carefully, and I found that in the `main` function of `RM`, there will be some other operations,

[GitHub] [hadoop] hadoop-yetus commented on pull request #5920: YARN-6476. Advanced Federation UI based on YARN UI v2.

2023-09-05 Thread via GitHub
hadoop-yetus commented on PR #5920: URL: https://github.com/apache/hadoop/pull/5920#issuecomment-1706506034 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on pull request #5920: YARN-6476. Advanced Federation UI based on YARN UI v2.

2023-09-05 Thread via GitHub
slfan1989 commented on PR #5920: URL: https://github.com/apache/hadoop/pull/5920#issuecomment-1706765303 @goiri Can you help review this PR again? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] slfan1989 commented on pull request #6011: YARN-11434. [Router] UGI conf doesn't read user overridden configurations on Router startup.

2023-09-05 Thread via GitHub
slfan1989 commented on PR #6011: URL: https://github.com/apache/hadoop/pull/6011#issuecomment-1706775668 @goiri Can you help review this pr? Thank you very much! This pr is based on the completion of YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup.

[GitHub] [hadoop] hadoop-yetus commented on pull request #6022: Expect100resolution

2023-09-05 Thread via GitHub
hadoop-yetus commented on PR #6022: URL: https://github.com/apache/hadoop/pull/6022#issuecomment-1706547824 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #6007: HADOOP-18870. CURATOR-599 change broke functionality introduced in HA…

2023-09-05 Thread via GitHub
hadoop-yetus commented on PR #6007: URL: https://github.com/apache/hadoop/pull/6007#issuecomment-1706842904 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5995: HADOOP-18818. Merge aws v2 upgrade feature branch into trunk

2023-09-05 Thread via GitHub
hadoop-yetus commented on PR #5995: URL: https://github.com/apache/hadoop/pull/5995#issuecomment-1706819927 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] KeeProMise commented on a diff in pull request #5990: HDFS-17166. RBF: Throwing NoNamenodesAvailableException for a long time, when failover

2023-09-02 Thread via GitHub
KeeProMise commented on code in PR #5990: URL: https://github.com/apache/hadoop/pull/5990#discussion_r1313862686 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java: ## @@ -599,6 +599,9 @@ public Object

[GitHub] [hadoop] KeeProMise commented on a diff in pull request #5990: HDFS-17166. RBF: Throwing NoNamenodesAvailableException for a long time, when failover

2023-09-02 Thread via GitHub
KeeProMise commented on code in PR #5990: URL: https://github.com/apache/hadoop/pull/5990#discussion_r1313862686 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java: ## @@ -599,6 +599,9 @@ public Object

[GitHub] [hadoop] hadoop-yetus commented on pull request #5920: YARN-6476. Advanced Federation UI based on YARN UI v2.

2023-09-02 Thread via GitHub
hadoop-yetus commented on PR #5920: URL: https://github.com/apache/hadoop/pull/5920#issuecomment-1703997064 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] dannytbecker opened a new pull request, #6018: HDFS-17178: Bootstrap Standby needs to handle RollingUpgrade

2023-09-03 Thread via GitHub
dannytbecker opened a new pull request, #6018: URL: https://github.com/apache/hadoop/pull/6018 ### Description of PR During rollingUpgrade, bootstrapStandby will fail with an exception due to different NameNodeLayoutVersions. We can ignore this safely during RollingUpgrade because

[GitHub] [hadoop] Hexiaoqiao commented on pull request #6002: HDFS-17172. Support FSNamesystemLock Parameters reconfigurable

2023-09-03 Thread via GitHub
Hexiaoqiao commented on PR #6002: URL: https://github.com/apache/hadoop/pull/6002#issuecomment-1704336062 @haiyang1987 Thanks for your works. I am confused why we do not enable it by default and remove to config, any performance impact? Thanks. -- This is an automated message from the

[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #6001: HADOOP-18851: Perfm improvement for ZKDT management

2023-09-03 Thread via GitHub
Hexiaoqiao commented on code in PR #6001: URL: https://github.com/apache/hadoop/pull/6001#discussion_r1314280986 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java: ## @@ -520,24 +525,32 @@

[GitHub] [hadoop] hadoop-yetus commented on pull request #6003: HADOOP-18869: [ABFS] Fixing Behavior of a File System APIs on root path

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #6003: URL: https://github.com/apache/hadoop/pull/6003#issuecomment-1704057809 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] Hexiaoqiao commented on pull request #5430: HDFS-16933. A race in SerialNumberMap will cause wrong owner, group and XATTR

2023-09-03 Thread via GitHub
Hexiaoqiao commented on PR #5430: URL: https://github.com/apache/hadoop/pull/5430#issuecomment-1704333917 @ZanderXu Thanks for your works. Great catch here! Do you take benchmark if any possible performance decrease here when parallel fsimage loading? -- This is an automated message from

[GitHub] [hadoop] Hexiaoqiao commented on pull request #5990: HDFS-17166. RBF: Throwing NoNamenodesAvailableException for a long time, when failover

2023-09-03 Thread via GitHub
Hexiaoqiao commented on PR #5990: URL: https://github.com/apache/hadoop/pull/5990#issuecomment-1704342943 Hi @goiri @simbadzina The latest CI report green here and you have approved this PR. If no more comments here, Please help to check in. Thanks. -- This is an automated message from

[GitHub] [hadoop] hadoop-yetus commented on pull request #6017: YARN-11558. Fix dependency convergence error on hbase2 profile.

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #6017: URL: https://github.com/apache/hadoop/pull/6017#issuecomment-1704349722 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #6004: HADOOP-18874: [ABFS] Adding Server returned request id in Exception Message thrown to caller.

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #6004: URL: https://github.com/apache/hadoop/pull/6004#issuecomment-1704063040 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5967: YARN-11435. [Router] FederationStateStoreFacade is not reinitialized with Router conf.

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #5967: URL: https://github.com/apache/hadoop/pull/5967#issuecomment-1704186867 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on pull request #5967: YARN-11435. [Router] FederationStateStoreFacade is not reinitialized with Router conf.

2023-09-03 Thread via GitHub
slfan1989 commented on PR #5967: URL: https://github.com/apache/hadoop/pull/5967#issuecomment-1704289235 @goiri Thank you very much for your help in reviewing the code! I will continue to follow up YARN-11433 And YARN-11434. -- This is an automated message from the Apache Git Service. To

[GitHub] [hadoop] KeeProMise commented on a diff in pull request #5990: HDFS-17166. RBF: Throwing NoNamenodesAvailableException for a long time, when failover

2023-09-03 Thread via GitHub
KeeProMise commented on code in PR #5990: URL: https://github.com/apache/hadoop/pull/5990#discussion_r1313860811 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/MockResolver.java: ## @@ -397,6 +397,11 @@ public List

[GitHub] [hadoop] slfan1989 merged pull request #5967: YARN-11435. [Router] FederationStateStoreFacade is not reinitialized with Router conf.

2023-09-03 Thread via GitHub
slfan1989 merged PR #5967: URL: https://github.com/apache/hadoop/pull/5967 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6015: HADOOP-18880. Add some rpc related metrics to Metrics.md

2023-09-03 Thread via GitHub
slfan1989 commented on code in PR #6015: URL: https://github.com/apache/hadoop/pull/6015#discussion_r1314246416 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/metrics/RpcMetrics.java: ## @@ -141,7 +141,7 @@ public static RpcMetrics create(Server

[GitHub] [hadoop] slfan1989 opened a new pull request, #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

2023-09-03 Thread via GitHub
slfan1989 opened a new pull request, #6016: URL: https://github.com/apache/hadoop/pull/6016 ### Description of PR JIRA: YARN-9048. Add znode hierarchy in Federation ZK State Store. ### How was this patch tested? ### For code changes: - [ ] Does the

[GitHub] [hadoop] haiyang1987 commented on a diff in pull request #6015: HADOOP-18880. Add some rpc related metrics to Metrics.md

2023-09-03 Thread via GitHub
haiyang1987 commented on code in PR #6015: URL: https://github.com/apache/hadoop/pull/6015#discussion_r1314247707 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/metrics/RpcMetrics.java: ## @@ -141,7 +141,7 @@ public static RpcMetrics create(Server

[GitHub] [hadoop] hadoop-yetus commented on pull request #5920: YARN-6476. Advanced Federation UI based on YARN UI v2.

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #5920: URL: https://github.com/apache/hadoop/pull/5920#issuecomment-1704309490 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] iwasakims commented on pull request #6017: YARN-11558. Fix dependency convergence error on hbase2 profile.

2023-09-03 Thread via GitHub
iwasakims commented on PR #6017: URL: https://github.com/apache/hadoop/pull/6017#issuecomment-1704304520 I ran tests of hadoop-yarn-server-timelineservice-hbase and hadoop-yarn-server-timelineservice-hbase-tests with `-Dhbase.profile=2.0`. ``` $

[GitHub] [hadoop] iwasakims opened a new pull request, #6017: YARN-11558. Fix dependency convergence error on hbase2 profile.

2023-09-03 Thread via GitHub
iwasakims opened a new pull request, #6017: URL: https://github.com/apache/hadoop/pull/6017 https://issues.apache.org/jira/browse/YARN-11558 Building with `-Dhbase.profile=2.0` fails due to dependency convergence error. ``` $ mvn clean install -DskipTests -DskipShade

[GitHub] [hadoop] hadoop-yetus commented on pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #6016: URL: https://github.com/apache/hadoop/pull/6016#issuecomment-1704323280 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] virajjasani commented on pull request #5893: HADOOP-18829. S3A prefetch LRU cache eviction metric

2023-09-03 Thread via GitHub
virajjasani commented on PR #5893: URL: https://github.com/apache/hadoop/pull/5893#issuecomment-1704373448 > Duration tracker for the removal of a block? done, thank you for this suggestion! -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop] zhtttylz commented on pull request #5992: HDFS-17168. Support getTrashRoots API in WebHDFS

2023-09-02 Thread via GitHub
zhtttylz commented on PR #5992: URL: https://github.com/apache/hadoop/pull/5992#issuecomment-1703889655 @slfan1989 Thank you for your assistance in reviewing the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5920: YARN-6476. Advanced Federation UI based on YARN UI v2.

2023-09-02 Thread via GitHub
slfan1989 commented on code in PR #5920: URL: https://github.com/apache/hadoop/pull/5920#discussion_r1314075142 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/Router.java: ## @@ -209,7 +214,48

[GitHub] [hadoop] zhuzilong2013 commented on pull request #5962: HDFS-16644.java.io.IOException Invalid token in javax.security.sasl.qop

2023-09-03 Thread via GitHub
zhuzilong2013 commented on PR #5962: URL: https://github.com/apache/hadoop/pull/5962#issuecomment-1704024711 This commit can help us avoid issues where the datanode fails to work properly after throwing exceptions with the 2.10.1 and 2.10.2 clients. I believe it's meaningful. @ayushtkn

[GitHub] [hadoop] hadoop-yetus commented on pull request #5990: HDFS-17166. RBF: Throwing NoNamenodesAvailableException for a long time, when failover

2023-09-02 Thread via GitHub
hadoop-yetus commented on PR #5990: URL: https://github.com/apache/hadoop/pull/5990#issuecomment-1703854738 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #6015: HADOOP-18880. Add some rpc related metrics to Metrics.md

2023-09-02 Thread via GitHub
hadoop-yetus commented on PR #6015: URL: https://github.com/apache/hadoop/pull/6015#issuecomment-1703882538 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] anujmodi2021 commented on a diff in pull request #6003: HADOOP-18869: [ABFS] Fixing Behavior of a File System APIs on root path

2023-09-03 Thread via GitHub
anujmodi2021 commented on code in PR #6003: URL: https://github.com/apache/hadoop/pull/6003#discussion_r1314145745 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemAttributes.java: ## @@ -110,4 +86,51 @@ public void

[GitHub] [hadoop] KeeProMise commented on a diff in pull request #5990: HDFS-17166. RBF: Throwing NoNamenodesAvailableException for a long time, when failover

2023-09-02 Thread via GitHub
KeeProMise commented on code in PR #5990: URL: https://github.com/apache/hadoop/pull/5990#discussion_r1313862686 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java: ## @@ -599,6 +599,9 @@ public Object

[GitHub] [hadoop] slfan1989 merged pull request #5957: YARN-6537. Running RM tests against the Router.

2023-09-03 Thread via GitHub
slfan1989 merged PR #5957: URL: https://github.com/apache/hadoop/pull/5957 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] haiyang1987 opened a new pull request, #6015: HADOOP-18880. Add some rpc related metrics to Metrics.md

2023-09-02 Thread via GitHub
haiyang1987 opened a new pull request, #6015: URL: https://github.com/apache/hadoop/pull/6015 ### Description of PR https://issues.apache.org/jira/browse/HADOOP-18880 Add some rpc related metrics to Metrics.md such as ``` DeferredRpcProcessingTime RpcResponseTime

[GitHub] [hadoop] anujmodi2021 commented on a diff in pull request #6003: HADOOP-18869: [ABFS] Fixing Behavior of a File System APIs on root path

2023-09-03 Thread via GitHub
anujmodi2021 commented on code in PR #6003: URL: https://github.com/apache/hadoop/pull/6003#discussion_r1314145876 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemCreate.java: ## @@ -146,6 +149,26 @@ public void

[GitHub] [hadoop] slfan1989 commented on pull request #5957: YARN-6537. Running RM tests against the Router.

2023-09-03 Thread via GitHub
slfan1989 commented on PR #5957: URL: https://github.com/apache/hadoop/pull/5957#issuecomment-1704038067 @goiri Thank you very much for your help in reviewing the code! I will continue to follow up other pr. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hadoop] KeeProMise commented on a diff in pull request #5990: HDFS-17166. RBF: Throwing NoNamenodesAvailableException for a long time, when failover

2023-09-02 Thread via GitHub
KeeProMise commented on code in PR #5990: URL: https://github.com/apache/hadoop/pull/5990#discussion_r1313860811 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/MockResolver.java: ## @@ -397,6 +397,11 @@ public List

[GitHub] [hadoop] KeeProMise commented on a diff in pull request #5990: HDFS-17166. RBF: Throwing NoNamenodesAvailableException for a long time, when failover

2023-09-02 Thread via GitHub
KeeProMise commented on code in PR #5990: URL: https://github.com/apache/hadoop/pull/5990#discussion_r1313862686 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java: ## @@ -599,6 +599,9 @@ public Object

[GitHub] [hadoop] virajjasani commented on a diff in pull request #5893: HADOOP-18829. S3A prefetch LRU cache eviction metric

2023-09-02 Thread via GitHub
virajjasani commented on code in PR #5893: URL: https://github.com/apache/hadoop/pull/5893#discussion_r1314086566 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/prefetch/SingleFilePerBlockCache.java: ## @@ -444,6 +444,7 @@ private void

[GitHub] [hadoop] anujmodi2021 commented on a diff in pull request #6003: HADOOP-18869: [ABFS] Fixing Behavior of a File System APIs on root path

2023-09-03 Thread via GitHub
anujmodi2021 commented on code in PR #6003: URL: https://github.com/apache/hadoop/pull/6003#discussion_r1314145683 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java: ## @@ -324,6 +325,12 @@ public FSDataOutputStream create(final

[GitHub] [hadoop] hadoop-yetus commented on pull request #6023: YARN-11562. [Federation] GPG Support Query Policies In Web.

2023-09-05 Thread via GitHub
hadoop-yetus commented on PR #6023: URL: https://github.com/apache/hadoop/pull/6023#issuecomment-1707022888 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #6001: HADOOP-18851: Performance improvement for DelegationTokenSecretManager.

2023-09-05 Thread via GitHub
hadoop-yetus commented on PR #6001: URL: https://github.com/apache/hadoop/pull/6001#issuecomment-1707078663 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] zhengchenyu commented on a diff in pull request #5975: YARN-8980. Mapreduce application container start fail after AM restart.

2023-09-03 Thread via GitHub
zhengchenyu commented on code in PR #5975: URL: https://github.com/apache/hadoop/pull/5975#discussion_r1314400717 ##

[GitHub] [hadoop] zhengchenyu commented on a diff in pull request #5975: YARN-8980. Mapreduce application container start fail after AM restart.

2023-09-03 Thread via GitHub
zhengchenyu commented on code in PR #5975: URL: https://github.com/apache/hadoop/pull/5975#discussion_r1314398962 ##

[GitHub] [hadoop] saxenapranav commented on a diff in pull request #6010: HADOOP-18873. ABFS: AbfsOutputStream doesnt close DataBlocks object.

2023-09-03 Thread via GitHub
saxenapranav commented on code in PR #6010: URL: https://github.com/apache/hadoop/pull/6010#discussion_r1314415846 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java: ## @@ -345,6 +345,7 @@ private void

[GitHub] [hadoop] mehakmeet commented on pull request #6000: HADOOP-18257. Merging and Parsing S3A audit logs into Avro format for analysis.

2023-09-03 Thread via GitHub
mehakmeet commented on PR #6000: URL: https://github.com/apache/hadoop/pull/6000#issuecomment-1704605738 CC: @steveloughran @mukund-thakur -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hadoop] hadoop-yetus commented on pull request #6018: HDFS-17178: BootstrapStandby needs to handle RollingUpgrade

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #6018: URL: https://github.com/apache/hadoop/pull/6018#issuecomment-1704557206 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] anujmodi2021 opened a new pull request, #6019: Hadoop 18872 tracing fix

2023-09-03 Thread via GitHub
anujmodi2021 opened a new pull request, #6019: URL: https://github.com/apache/hadoop/pull/6019 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g.

[GitHub] [hadoop] hadoop-yetus commented on pull request #6018: HDFS-17178: BootstrapStandby needs to handle RollingUpgrade

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #6018: URL: https://github.com/apache/hadoop/pull/6018#issuecomment-1704397100 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] zhengchenyu commented on a diff in pull request #5975: YARN-8980. Mapreduce application container start fail after AM restart.

2023-09-03 Thread via GitHub
zhengchenyu commented on code in PR #5975: URL: https://github.com/apache/hadoop/pull/5975#discussion_r1314398549 ##

[GitHub] [hadoop] anujmodi2021 commented on a diff in pull request #5881: Hadoop-18759: [ABFS][Backoff-Optimization] Have a Static retry policy for connection timeout.

2023-09-03 Thread via GitHub
anujmodi2021 commented on code in PR #5881: URL: https://github.com/apache/hadoop/pull/5881#discussion_r1308427326 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/StaticRetryPolicy.java: ## @@ -0,0 +1,68 @@ +/** + * Licensed to the Apache

[GitHub] [hadoop] hadoop-yetus commented on pull request #5893: HADOOP-18829. S3A prefetch LRU cache eviction metrics

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #5893: URL: https://github.com/apache/hadoop/pull/5893#issuecomment-1704650805 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] zhengchenyu commented on a diff in pull request #5975: YARN-8980. Mapreduce application container start fail after AM restart.

2023-09-04 Thread via GitHub
zhengchenyu commented on code in PR #5975: URL: https://github.com/apache/hadoop/pull/5975#discussion_r1314398549 ##

[GitHub] [hadoop] zhengchenyu commented on a diff in pull request #5975: YARN-8980. Mapreduce application container start fail after AM restart.

2023-09-03 Thread via GitHub
zhengchenyu commented on code in PR #5975: URL: https://github.com/apache/hadoop/pull/5975#discussion_r1314398962 ##

[GitHub] [hadoop] hadoop-yetus commented on pull request #6019: HADOOP-18872: [ABFS] [BugFix] Misreporting Retry Count for Sub-sequential and Parallel Operations

2023-09-04 Thread via GitHub
hadoop-yetus commented on PR #6019: URL: https://github.com/apache/hadoop/pull/6019#issuecomment-1704696088 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] virajjasani commented on pull request #5893: HADOOP-18829. S3A prefetch LRU cache eviction metrics

2023-09-03 Thread via GitHub
virajjasani commented on PR #5893: URL: https://github.com/apache/hadoop/pull/5893#issuecomment-1704478561 `us-west-2`: `mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Dscale -Dprefetch` Checkstyle complains about 8th param added to CachingBlockManager c'tor. Shall we

[GitHub] [hadoop] Aakansha99 opened a new pull request, #6020: Oss/hadoop java17 runtime feature branch prerequisite

2023-09-03 Thread via GitHub
Aakansha99 opened a new pull request, #6020: URL: https://github.com/apache/hadoop/pull/6020 ### Description of PR The latest Hadoop version 3.3.3 which supports Java 8 compile and [Java 8, Java 11](https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Java+Versions)

[GitHub] [hadoop] hadoop-yetus commented on pull request #5893: HADOOP-18829. S3A prefetch LRU cache eviction metrics

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #5893: URL: https://github.com/apache/hadoop/pull/5893#issuecomment-1704428844 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #6013: HDFS-17167. Add config to startup NameNode as Observer

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #6013: URL: https://github.com/apache/hadoop/pull/6013#issuecomment-1704445299 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] dannytbecker commented on pull request #6013: HDFS-17167. Add config to startup NameNode as Observer

2023-09-03 Thread via GitHub
dannytbecker commented on PR #6013: URL: https://github.com/apache/hadoop/pull/6013#issuecomment-1704453488 @simbadzina @goiri Looks like the test failure is unrelated. The test I added, “TestObserverNode#testConfigStartup”, was fixed. -- This is an automated message from the Apache Git

[GitHub] [hadoop] Hexiaoqiao merged pull request #5430: HDFS-16933. A race in SerialNumberMap will cause wrong owner, group and XATTR

2023-09-05 Thread via GitHub
Hexiaoqiao merged PR #5430: URL: https://github.com/apache/hadoop/pull/5430 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] Hexiaoqiao commented on pull request #5430: HDFS-16933. A race in SerialNumberMap will cause wrong owner, group and XATTR

2023-09-05 Thread via GitHub
Hexiaoqiao commented on PR #5430: URL: https://github.com/apache/hadoop/pull/5430#issuecomment-1707561894 Committed to trunk. Thanks @ZanderXu for your contribution! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hadoop] simbadzina commented on a diff in pull request #6013: HDFS-17167. Add config to startup NameNode as Observer

2023-09-05 Thread via GitHub
simbadzina commented on code in PR #6013: URL: https://github.com/apache/hadoop/pull/6013#discussion_r1316761156 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java: ## @@ -1161,11 +1163,17 @@ private void

[GitHub] [hadoop] simbadzina commented on a diff in pull request #6013: HDFS-17167. Add config to startup NameNode as Observer

2023-09-05 Thread via GitHub
simbadzina commented on code in PR #6013: URL: https://github.com/apache/hadoop/pull/6013#discussion_r1316764347 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java: ## @@ -1161,11 +1163,17 @@ private void

[GitHub] [hadoop] zhangxiping1 commented on pull request #5921: HDFS-17138 RBF: We changed the hadoop.security.auth_to_local configur…

2023-09-06 Thread via GitHub
zhangxiping1 commented on PR #5921: URL: https://github.com/apache/hadoop/pull/5921#issuecomment-1707729841 @Hexiaoqiao @goiri Does this PR still need to be improved? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] simbadzina commented on a diff in pull request #6013: HDFS-17167. Add config to startup NameNode as Observer

2023-09-05 Thread via GitHub
simbadzina commented on code in PR #6013: URL: https://github.com/apache/hadoop/pull/6013#discussion_r1316506127 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java: ## @@ -1161,11 +1163,17 @@ private void

[GitHub] [hadoop] goiri commented on a diff in pull request #6012: YARN-11433. Router's main() should support generic options.

2023-09-05 Thread via GitHub
goiri commented on code in PR #6012: URL: https://github.com/apache/hadoop/pull/6012#discussion_r1316558450 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/Router.java: ## @@ -245,18 +247,26 @@

  1   2   3   4   5   6   7   8   9   10   >