Here are some summaries about the first phase: 1. There are no big changes in this phase 2. This phase just uses FS lock and BM lock to replace the original global lock 3. It's useful to improve the performance, since some operations just need to hold FS lock or BM lock instead of the global lock 4. This feature is turned off by default, you can enable it by setting dfs.namenode.lock.model.provider.class to org.apache.hadoop.hdfs.server.namenode.fgl.FineGrainedFSNamesystemLock 5. This phase is very import for the ongoing development of the entire FGL
Here I would like to express my special thanks to @kokonguyen191 and @yuanboliu for their contributions. And you are also welcome to join us and complete it together. On Wed, 24 Apr 2024 at 14:54, ZanderXu <zande...@apache.org> wrote: > Hi everyone > > All subtasks of the first phase of the FGL have been completed and I plan > to merge them into the trunk and start the second phase based on the trunk. > > Here is the PR that used to merge the first phases into trunk: > https://github.com/apache/hadoop/pull/6762 > Here is the ticket: https://issues.apache.org/jira/browse/HDFS-17384 > > I hope you can help to review this PR when you are available and give some > ideas. > > > HDFS-17385 <https://issues.apache.org/jira/browse/HDFS-17385> is used for > the second phase and I have created some subtasks to describe solutions for > some problems, such as: snapshot, getListing, quota. > You are welcome to join us to complete it together. > > > ---------- Forwarded message --------- > From: Zengqiang XU <zande...@apache.org> > Date: Fri, 2 Feb 2024 at 11:07 > Subject: Discussion about NameNode Fine-grained locking > To: <hdfs-dev@hadoop.apache.org> > Cc: Zengqiang XU <xuzengqiang5...@gmail.com> > > > Hi everyone > > I have started a discussion about NameNode Fine-grained Locking to improve > performance of write operations in NameNode. > > I started this discussion again for serval main reasons: > 1. We have implemented it and gained nearly 7x performance improvement in > our prod environment > 2. Many other companies made similar improvements based on their internal > branch. > 3. This topic has been discussed for a long time, but still without any > results. > > I hope we can push this important improvement in the community so that all > end-users can enjoy this significant improvement. > > I'd really appreciate you can join in and work with me to push this > feature forward. > > Thanks very much. > > Ticket: HDFS-17366 <https://issues.apache.org/jira/browse/HDFS-17366> > Design: NameNode Fine-grained locking based on directory tree > <https://docs.google.com/document/d/1X499gHxT0WSU1fj8uo4RuF3GqKxWkWXznXx4tspTBLY/edit?usp=sharing> >
