[jira] [Commented] (HDDS-1672) Improve locking in OzoneManager
[ https://issues.apache.org/jira/browse/HDDS-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16875228#comment-16875228 ] Hudson commented on HDDS-1672: -- FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #16837 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/16837/]) HDDS-1672. Improve locking in OzoneManager. (#1016) (github: rev 49c5e8ac249981b533763d1523e72872748e3f79) * (edit) hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMMetadataManager.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/PrefixManagerImpl.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/volume/OMVolumeSetQuotaRequest.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequest.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyRenameRequest.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetadataManagerImpl.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/VolumeManagerImpl.java * (delete) hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerLock.java * (edit) hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/lock/OzoneManagerLock.java * (edit) hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/S3SecretManagerImpl.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyDeleteRequest.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/S3BucketManagerImpl.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java * (delete) hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OzoneManagerLock.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCreateRequest.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/volume/OMVolumeSetOwnerRequest.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/volume/OMVolumeCreateRequest.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/S3BucketManager.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/volume/OMVolumeDeleteRequest.java * (edit) hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/lock/TestOzoneManagerLock.java * (edit) hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/lock/OzoneManagerLockUtil.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/BucketManagerImpl.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketDeleteRequest.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketSetPropertyRequest.java > Improve locking in OzoneManager > --- > > Key: HDDS-1672 > URL: https://issues.apache.org/jira/browse/HDDS-1672 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Manager >Affects Versions: 0.4.0 >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Attachments: Ozone Locks in OM.pdf > > Time Spent: 10.5h > Remaining Estimate: 0h > > In this Jira, we shall follow the new lock ordering. In this way, in volume > requests we can solve the issue of acquire/release/reacquire problem. And few > bugs in the current implementation of S3Bucket/Volume operations. > > Currently after acquiring volume lock, we cannot acquire user lock. > This is causing an issue in Volume request implementation, > acquire/release/reacquire volume lock. > > Case of Delete Volume Request: > # Acquire volume lock. > # Get Volume Info from DB > # Release Volume lock. (We are releasing the lock, because while acquiring > volume lock, we cannot acquire user lock0 > # Get owner from volume Info read from DB > # Acquire owner lock > # Acquire volume lock > # Do delete logic > # release volume lock > # release user lock > > We can avoid this acquire/release/reacquire lock issue by making volume lock > as low weight. > > In this way, the above deleteVolume request will change as below > # Acquire volume lock > # Get Volume Info from DB > # Get owner from volume Info read from DB > # Acquire owner lock > # Do delete logic > # release owner lock > # release
[jira] [Commented] (HDDS-1672) Improve locking in OzoneManager
[ https://issues.apache.org/jira/browse/HDDS-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16872506#comment-16872506 ] Bharat Viswanadham commented on HDDS-1672: -- Hi [~anu] The new LockManager is checked in as part of HDDS-1723. I will use this Jira to use new API's provided in HDDS-1723. > Improve locking in OzoneManager > --- > > Key: HDDS-1672 > URL: https://issues.apache.org/jira/browse/HDDS-1672 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Manager >Affects Versions: 0.4.0 >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Fix For: 0.4.1 > > Attachments: Ozone Locks in OM.pdf > > Time Spent: 7h 10m > Remaining Estimate: 0h > > In this Jira, we shall follow the new lock ordering. In this way, in volume > requests we can solve the issue of acquire/release/reacquire problem. And few > bugs in the current implementation of S3Bucket/Volume operations. > > Currently after acquiring volume lock, we cannot acquire user lock. > This is causing an issue in Volume request implementation, > acquire/release/reacquire volume lock. > > Case of Delete Volume Request: > # Acquire volume lock. > # Get Volume Info from DB > # Release Volume lock. (We are releasing the lock, because while acquiring > volume lock, we cannot acquire user lock0 > # Get owner from volume Info read from DB > # Acquire owner lock > # Acquire volume lock > # Do delete logic > # release volume lock > # release user lock > > We can avoid this acquire/release/reacquire lock issue by making volume lock > as low weight. > > In this way, the above deleteVolume request will change as below > # Acquire volume lock > # Get Volume Info from DB > # Get owner from volume Info read from DB > # Acquire owner lock > # Do delete logic > # release owner lock > # release volume lock. > Same issue is seen with SetOwner for Volume request also. > During HDDS-1620 [~arp] brought up this issue. > I am proposing the above solution to solve this issue. Any other > idea/suggestions are welcome. > This also resolves a bug in setOwner for Volume request. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-1672) Improve locking in OzoneManager
[ https://issues.apache.org/jira/browse/HDDS-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871930#comment-16871930 ] Bharat Viswanadham commented on HDDS-1672: -- Canceled the patch. As we are coming up with new OzoneManagerLock.java. Once HDDS-1723 gets checked in, will work on this, using new API's implemented in HDDS-1723. > Improve locking in OzoneManager > --- > > Key: HDDS-1672 > URL: https://issues.apache.org/jira/browse/HDDS-1672 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Manager >Affects Versions: 0.4.0 >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Attachments: Ozone Locks in OM.pdf > > Time Spent: 7h 10m > Remaining Estimate: 0h > > In this Jira, we shall follow the new lock ordering. In this way, in volume > requests we can solve the issue of acquire/release/reacquire problem. And few > bugs in the current implementation of S3Bucket/Volume operations. > > Currently after acquiring volume lock, we cannot acquire user lock. > This is causing an issue in Volume request implementation, > acquire/release/reacquire volume lock. > > Case of Delete Volume Request: > # Acquire volume lock. > # Get Volume Info from DB > # Release Volume lock. (We are releasing the lock, because while acquiring > volume lock, we cannot acquire user lock0 > # Get owner from volume Info read from DB > # Acquire owner lock > # Acquire volume lock > # Do delete logic > # release volume lock > # release user lock > > We can avoid this acquire/release/reacquire lock issue by making volume lock > as low weight. > > In this way, the above deleteVolume request will change as below > # Acquire volume lock > # Get Volume Info from DB > # Get owner from volume Info read from DB > # Acquire owner lock > # Do delete logic > # release owner lock > # release volume lock. > Same issue is seen with SetOwner for Volume request also. > During HDDS-1620 [~arp] brought up this issue. > I am proposing the above solution to solve this issue. Any other > idea/suggestions are welcome. > This also resolves a bug in setOwner for Volume request. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
