[GitHub] [hadoop] anuengineer commented on issue #949: HDDS-1672. OzoneManager Lock change the volumeLock weight to 0
anuengineer commented on issue #949: HDDS-1672. OzoneManager Lock change the volumeLock weight to 0 URL: https://github.com/apache/hadoop/pull/949#issuecomment-501443153 > @anuengineer could you please clarify what this table looks like? This is the first time in 20+ years of writing multi-threaded code that I have have heard of a locking conflict resolution table. I have only heard of locking order so far. Something like Lock Levels - User=0, Volume=1,Bucket=2, Key =3 Create Volume -> Locking Order -> Lock User, LockVolume , Release Volume, Release User Update Volume, Locking Order -> L(V), R(V) L(U), L(V), R(V), R(U). For each path, you know what the locking order is. We have a lock hierarchy very clearly defined today, changing that order means that you need to make sure no other path which is taking those locks are now violating the proposed order. We need to manually list out each lock path in the OM to move user lock from level O to level 1. In other words, changing lock hierarchy in any code path is a non-trivial. This process will prove that our hierarchy itself is not buggy, and you don't run into this same problem is some other path, since you have optimized code for this path. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] anuengineer commented on issue #949: HDDS-1672. OzoneManager Lock change the volumeLock weight to 0
anuengineer commented on issue #949: HDDS-1672. OzoneManager Lock change the volumeLock weight to 0 URL: https://github.com/apache/hadoop/pull/949#issuecomment-501388383 > Thank You @anuengineer for the comments. > Could you provide a table which has been written when it is first designed? I can take a look at it, and >see what is meant by locking conflict resolution table, and see how to come up with such a table, and >see if this is breaking anything. I will try to find it, @nandakumar131 and I worked on this together, I think Nanda wrote that document. These are the kind of docs I wish were committed it the design repo so that we can find them when we need. I don't think that doc was posted to Apache, But I will see if I can find it, else we will have to rebuild that lock conflict table. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] anuengineer commented on issue #949: HDDS-1672. OzoneManager Lock change the volumeLock weight to 0
anuengineer commented on issue #949: HDDS-1672. OzoneManager Lock change the volumeLock weight to 0 URL: https://github.com/apache/hadoop/pull/949#issuecomment-501382532 Unless I am completely out of my mind, this is a dangerous patch to commit. We need to prove that taking user locks after volume locks is consistent across all paths. Right now, we think of user/volume/bucket/key as a logical hierarchy. Yes, there are cases where you only have volume and you have to look back up the path, and in that case, you might have to release a lock already taken. But that is not an excuse to change the locking hierarchy if you want to do that, You need to write down a locking conflict resolution table and prove that all paths are consistent. That is what we did when we first developed this locking hierarchy table, Yes, it is a bit of work now, but will save us countless hours of work and pain in the future. Let us not commit this unless we have such a table. As I said, it might be that this is correct, but this is a place where we have to be sure that this is correct. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
