shuaiqig commented on PR #6236: URL: https://github.com/apache/hadoop/pull/6236#issuecomment-1835365341
> > When rollEditLog() is called, ANN writes to seen_txid in both the dfs.namenode.name.dir and the dfs.namenode.edits.dir (regardless of whether they are isolated or not), using a write lock . If the ioutil is high, it will take a long time to write the small file seen_txid, so indirectly cause ANN to hold the write lock for a long time. > > Back to this PR. For HA-mode cluster, if we set the same storage device for both `dfs.namenode.name.dir` and `dfs.namenode.edits.dir`, it could lead high load of this storage, especially for large cluster and impact performance of ANN. [HDFS-12733](https://issues.apache.org/jira/browse/HDFS-12733) try to disable local edit for HA-mode with shared edit dirs, which proposed years ago. (NOTE: this is draft patch and not checkin to trunk, could not checkin smoothy now, it need to review carefully if reference.) Hope it could solve this issue. Thanks. I have set the different path for `dfs.namenode.name.dir` and `dfs.namenode.edits.dir`, but they are still on the same storage device. I will try to set different storage devices for them later. Thanks for your help. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
