[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[ https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=223450=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-223450 ] ASF GitHub Bot logged work on HDDS-1379: Author: ASF GitHub Bot Created on: 05/Apr/19 05:29 Start Date: 05/Apr/19 05:29 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #689: HDDS-1379. Convert all OM Volume related operations to HA model. URL: https://github.com/apache/hadoop/pull/689#issuecomment-480153136 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 0 | Docker mode activated. | | -1 | patch | 9 | https://github.com/apache/hadoop/pull/689 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hadoop/pull/689 | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-689/7/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 223450) Time Spent: 4h 50m (was: 4h 40m) > Convert all OM Volume related operations to HA model > > > Key: HDDS-1379 > URL: https://issues.apache.org/jira/browse/HDDS-1379 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Fix For: 0.5.0 > > Time Spent: 4h 50m > Remaining Estimate: 0h > > In this jira, we shall convert all OM related operations to OM HA model, > which is a 2 step. > # StartTransaction, where we validate request and check for any errors and > return the response. > # ApplyTransaction, where original OM request will have a response which > needs to be applied to OM DB. This step is just to apply response to Om DB. > In this way, all requests which are failed with like volume not found or some > conditions which i have not satisfied like when deleting volume should be > empty, these all will be executed during startTransaction, and if it fails > these requests will not be written to raft log also. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[ https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=223429=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-223429 ] ASF GitHub Bot logged work on HDDS-1379: Author: ASF GitHub Bot Created on: 05/Apr/19 04:09 Start Date: 05/Apr/19 04:09 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on pull request #689: HDDS-1379. Convert all OM Volume related operations to HA model. URL: https://github.com/apache/hadoop/pull/689 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 223429) Time Spent: 4h 40m (was: 4.5h) > Convert all OM Volume related operations to HA model > > > Key: HDDS-1379 > URL: https://issues.apache.org/jira/browse/HDDS-1379 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Time Spent: 4h 40m > Remaining Estimate: 0h > > In this jira, we shall convert all OM related operations to OM HA model, > which is a 2 step. > # StartTransaction, where we validate request and check for any errors and > return the response. > # ApplyTransaction, where original OM request will have a response which > needs to be applied to OM DB. This step is just to apply response to Om DB. > In this way, all requests which are failed with like volume not found or some > conditions which i have not satisfied like when deleting volume should be > empty, these all will be executed during startTransaction, and if it fails > these requests will not be written to raft log also. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[ https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=223427=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-223427 ] ASF GitHub Bot logged work on HDDS-1379: Author: ASF GitHub Bot Created on: 05/Apr/19 04:06 Start Date: 05/Apr/19 04:06 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #689: HDDS-1379. Convert all OM Volume related operations to HA model. URL: https://github.com/apache/hadoop/pull/689#issuecomment-480141525 For TestScmChillMode we have a jira open to fix the issue. For other the failure is due to cluster startup failure to Failed to bind. I think it is a random failure. This is not related to this patch. I will commit this. Thank You @arp7 for the review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 223427) Time Spent: 4.5h (was: 4h 20m) > Convert all OM Volume related operations to HA model > > > Key: HDDS-1379 > URL: https://issues.apache.org/jira/browse/HDDS-1379 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Time Spent: 4.5h > Remaining Estimate: 0h > > In this jira, we shall convert all OM related operations to OM HA model, > which is a 2 step. > # StartTransaction, where we validate request and check for any errors and > return the response. > # ApplyTransaction, where original OM request will have a response which > needs to be applied to OM DB. This step is just to apply response to Om DB. > In this way, all requests which are failed with like volume not found or some > conditions which i have not satisfied like when deleting volume should be > empty, these all will be executed during startTransaction, and if it fails > these requests will not be written to raft log also. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[ https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=223426=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-223426 ] ASF GitHub Bot logged work on HDDS-1379: Author: ASF GitHub Bot Created on: 05/Apr/19 04:05 Start Date: 05/Apr/19 04:05 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #689: HDDS-1379. Convert all OM Volume related operations to HA model. URL: https://github.com/apache/hadoop/pull/689#issuecomment-480141525 For TestScmChillMode we have a jira open to fix the issue. For other the failure is due to cluster startup failure to Failed to bind. I think it is a random failure. This is not related to this patch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 223426) Time Spent: 4h 20m (was: 4h 10m) > Convert all OM Volume related operations to HA model > > > Key: HDDS-1379 > URL: https://issues.apache.org/jira/browse/HDDS-1379 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Time Spent: 4h 20m > Remaining Estimate: 0h > > In this jira, we shall convert all OM related operations to OM HA model, > which is a 2 step. > # StartTransaction, where we validate request and check for any errors and > return the response. > # ApplyTransaction, where original OM request will have a response which > needs to be applied to OM DB. This step is just to apply response to Om DB. > In this way, all requests which are failed with like volume not found or some > conditions which i have not satisfied like when deleting volume should be > empty, these all will be executed during startTransaction, and if it fails > these requests will not be written to raft log also. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[ https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=223425=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-223425 ] ASF GitHub Bot logged work on HDDS-1379: Author: ASF GitHub Bot Created on: 05/Apr/19 04:00 Start Date: 05/Apr/19 04:00 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #689: HDDS-1379. Convert all OM Volume related operations to HA model. URL: https://github.com/apache/hadoop/pull/689#issuecomment-480140918 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 25 | Docker mode activated. | ||| _ Prechecks _ | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 26 | Maven dependency ordering for branch | | +1 | mvninstall | 1016 | trunk passed | | +1 | compile | 102 | trunk passed | | +1 | checkstyle | 45 | trunk passed | | +1 | mvnsite | 97 | trunk passed | | +1 | shadedclient | 732 | branch has no errors when building and testing our client artifacts. | | 0 | findbugs | 0 | Skipped patched modules with no Java source: hadoop-ozone/integration-test | | +1 | findbugs | 93 | trunk passed | | +1 | javadoc | 66 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 11 | Maven dependency ordering for patch | | +1 | mvninstall | 92 | the patch passed | | +1 | compile | 99 | the patch passed | | +1 | cc | 99 | the patch passed | | +1 | javac | 99 | the patch passed | | +1 | checkstyle | 21 | the patch passed | | +1 | mvnsite | 80 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 668 | patch has no errors when building and testing our client artifacts. | | 0 | findbugs | 0 | Skipped patched modules with no Java source: hadoop-ozone/integration-test | | +1 | findbugs | 109 | the patch passed | | -1 | javadoc | 34 | hadoop-ozone_common generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | ||| _ Other Tests _ | | +1 | unit | 32 | common in the patch passed. | | +1 | unit | 42 | ozone-manager in the patch passed. | | -1 | unit | 1308 | integration-test in the patch failed. | | +1 | asflicense | 32 | The patch does not generate ASF License warnings. | | | | 4778 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.om.TestOzoneManagerHA | | | hadoop.ozone.om.TestScmChillMode | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-689/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/689 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux d24b809b7c10 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / d2141ef | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_191 | | findbugs | v3.1.0-RC1 | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-689/6/artifact/out/diff-javadoc-javadoc-hadoop-ozone_common.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-689/6/artifact/out/patch-unit-hadoop-ozone_integration-test.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-689/6/testReport/ | | Max. process+thread count | 4263 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/common hadoop-ozone/ozone-manager hadoop-ozone/integration-test U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-689/6/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 223425) Time Spent: 4h 10m (was: 4h) > Convert all OM Volume related operations to HA model > > > Key: HDDS-1379 > URL: https://issues.apache.org/jira/browse/HDDS-1379 > Project: Hadoop Distributed Data Store >
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[ https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=223386=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-223386 ] ASF GitHub Bot logged work on HDDS-1379: Author: ASF GitHub Bot Created on: 05/Apr/19 00:32 Start Date: 05/Apr/19 00:32 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #689: HDDS-1379. Convert all OM Volume related operations to HA model. URL: https://github.com/apache/hadoop/pull/689#issuecomment-480109833 Fixed test failure which is related to this patch. Other issues are not related to this patch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 223386) Time Spent: 4h (was: 3h 50m) > Convert all OM Volume related operations to HA model > > > Key: HDDS-1379 > URL: https://issues.apache.org/jira/browse/HDDS-1379 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Time Spent: 4h > Remaining Estimate: 0h > > In this jira, we shall convert all OM related operations to OM HA model, > which is a 2 step. > # StartTransaction, where we validate request and check for any errors and > return the response. > # ApplyTransaction, where original OM request will have a response which > needs to be applied to OM DB. This step is just to apply response to Om DB. > In this way, all requests which are failed with like volume not found or some > conditions which i have not satisfied like when deleting volume should be > empty, these all will be executed during startTransaction, and if it fails > these requests will not be written to raft log also. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[ https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=223278=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-223278 ] ASF GitHub Bot logged work on HDDS-1379: Author: ASF GitHub Bot Created on: 04/Apr/19 21:50 Start Date: 04/Apr/19 21:50 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #689: HDDS-1379. Convert all OM Volume related operations to HA model. URL: https://github.com/apache/hadoop/pull/689#issuecomment-480076561 Thank You @arp7 for review. Posted a patch to fix asf license errors and Jenkins find bug issue. I will commit this after a clean Jenkins run. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 223278) Time Spent: 3h 50m (was: 3h 40m) > Convert all OM Volume related operations to HA model > > > Key: HDDS-1379 > URL: https://issues.apache.org/jira/browse/HDDS-1379 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Time Spent: 3h 50m > Remaining Estimate: 0h > > In this jira, we shall convert all OM related operations to OM HA model, > which is a 2 step. > # StartTransaction, where we validate request and check for any errors and > return the response. > # ApplyTransaction, where original OM request will have a response which > needs to be applied to OM DB. This step is just to apply response to Om DB. > In this way, all requests which are failed with like volume not found or some > conditions which i have not satisfied like when deleting volume should be > empty, these all will be executed during startTransaction, and if it fails > these requests will not be written to raft log also. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[ https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=223250=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-223250 ] ASF GitHub Bot logged work on HDDS-1379: Author: ASF GitHub Bot Created on: 04/Apr/19 21:04 Start Date: 04/Apr/19 21:04 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #689: HDDS-1379. Convert all OM Volume related operations to HA model. URL: https://github.com/apache/hadoop/pull/689#issuecomment-480063273 Hi @arp7 Thanks for the review. Fixed Jenkins issues and addressed review comments. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 223250) Time Spent: 3h 40m (was: 3.5h) > Convert all OM Volume related operations to HA model > > > Key: HDDS-1379 > URL: https://issues.apache.org/jira/browse/HDDS-1379 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > In this jira, we shall convert all OM related operations to OM HA model, > which is a 2 step. > # StartTransaction, where we validate request and check for any errors and > return the response. > # ApplyTransaction, where original OM request will have a response which > needs to be applied to OM DB. This step is just to apply response to Om DB. > In this way, all requests which are failed with like volume not found or some > conditions which i have not satisfied like when deleting volume should be > empty, these all will be executed during startTransaction, and if it fails > these requests will not be written to raft log also. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=223249=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-223249
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 04/Apr/19 21:03
Start Date: 04/Apr/19 21:03
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #689:
HDDS-1379. Convert all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r272366957
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##
@@ -1594,6 +1598,107 @@ public void createVolume(OmVolumeArgs args) throws
IOException {
}
}
+ @Override
+ public VolumeList startCreateVolume(OmVolumeArgs args) throws IOException {
+try {
+ // TODO: Need to add metrics and Audit log for HA requests
Review comment:
Filed https://issues.apache.org/jira/browse/HDDS-1392
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 223249)
Time Spent: 3.5h (was: 3h 20m)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=223247=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-223247
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 04/Apr/19 21:01
Start Date: 04/Apr/19 21:01
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #689:
HDDS-1379. Convert all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r272366237
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##
@@ -1594,6 +1598,107 @@ public void createVolume(OmVolumeArgs args) throws
IOException {
}
}
+ @Override
+ public VolumeList startCreateVolume(OmVolumeArgs args) throws IOException {
+try {
+ // TODO: Need to add metrics and Audit log for HA requests
+ if(isAclEnabled) {
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 223247)
Time Spent: 3h 20m (was: 3h 10m)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=223246=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-223246
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 04/Apr/19 21:01
Start Date: 04/Apr/19 21:01
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #689:
HDDS-1379. Convert all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r272366183
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##
@@ -1594,6 +1598,107 @@ public void createVolume(OmVolumeArgs args) throws
IOException {
}
}
+ @Override
+ public VolumeList startCreateVolume(OmVolumeArgs args) throws IOException {
+try {
+ // TODO: Need to add metrics and Audit log for HA requests
+ if(isAclEnabled) {
+checkAcls(ResourceType.VOLUME, StoreType.OZONE,
+ACLType.CREATE, args.getVolume(), null, null);
+ }
+ VolumeList volumeList = volumeManager.createVolume(args);
+ return volumeList;
+} catch (Exception ex) {
+ throw ex;
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 223246)
Time Spent: 3h 10m (was: 3h)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222715=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222715
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 04/Apr/19 00:00
Start Date: 04/Apr/19 00:00
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #689:
HDDS-1379. Convert all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271976741
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##
@@ -1594,6 +1598,107 @@ public void createVolume(OmVolumeArgs args) throws
IOException {
}
}
+ @Override
+ public VolumeList startCreateVolume(OmVolumeArgs args) throws IOException {
+try {
+ // TODO: Need to add metrics and Audit log for HA requests
+ if(isAclEnabled) {
Review comment:
Will update it
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 222715)
Time Spent: 3h (was: 2h 50m)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222714=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222714
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 04/Apr/19 00:00
Start Date: 04/Apr/19 00:00
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #689:
HDDS-1379. Convert all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271976703
##
File path:
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmVolumeOwnerChangeResponse.java
##
@@ -0,0 +1,56 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.helpers;
+
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos
+.VolumeList;
+
+/**
+ * OM response for owner change request for a ozone volume.
+ */
+public class OmVolumeOwnerChangeResponse {
+ private VolumeList originalOwnerVolumeList;
Review comment:
I think you have figured out this, but adding my response here.
Added these fields because during applyTransaction we don't want to read
OMDB, in applyTransaction, it is just applied to OM DB. (Like put call, commit
batch call). If we don't return these values, we need to read OM DB again in
applyTransaction.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 222714)
Time Spent: 2h 50m (was: 2h 40m)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222713=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222713
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 03/Apr/19 23:58
Start Date: 03/Apr/19 23:58
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #689:
HDDS-1379. Convert all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271976281
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##
@@ -1594,6 +1598,107 @@ public void createVolume(OmVolumeArgs args) throws
IOException {
}
}
+ @Override
+ public VolumeList startCreateVolume(OmVolumeArgs args) throws IOException {
+try {
+ // TODO: Need to add metrics and Audit log for HA requests
+ if(isAclEnabled) {
+checkAcls(ResourceType.VOLUME, StoreType.OZONE,
+ACLType.CREATE, args.getVolume(), null, null);
+ }
+ VolumeList volumeList = volumeManager.createVolume(args);
+ return volumeList;
+} catch (Exception ex) {
+ throw ex;
Review comment:
Will update it
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 222713)
Time Spent: 2h 40m (was: 2.5h)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222712=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222712
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 03/Apr/19 23:58
Start Date: 03/Apr/19 23:58
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #689:
HDDS-1379. Convert all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271976232
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/S3BucketManagerImpl.java
##
@@ -166,6 +170,10 @@ public boolean createOzoneVolumeIfNeeded(String userName)
.setVolume(ozoneVolumeName)
.setQuotaInBytes(OzoneConsts.MAX_QUOTA_IN_BYTES)
.build();
+ if (isRatisEnabled) {
Review comment:
S3Bucket create internally calls createVolume. As now we have not separated
s3 createbucket into 2 phase, we need to do this (In applyTransaction
createBucket is called now). As createVolume when ratis is enabled it does not
apply to OM DB. So, we need to call apply also here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 222712)
Time Spent: 2.5h (was: 2h 20m)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222710=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222710
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 03/Apr/19 23:56
Start Date: 03/Apr/19 23:56
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #689:
HDDS-1379. Convert all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271975878
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/OzoneManagerRequestHandler.java
##
@@ -363,6 +365,199 @@ public OMResponse handle(OMRequest request) {
return responseBuilder.build();
}
+ @Override
+ public OMRequest handleStartTransaction(OMRequest omRequest)
+ throws IOException {
+LOG.debug("Received OMRequest: {}, ", omRequest);
+Type cmdType = omRequest.getCmdType();
+OMRequest newOmRequest = null;
+try {
+ switch (cmdType) {
+ case CreateVolume:
+newOmRequest = handleCreateVolumeStart(omRequest);
+break;
+ case SetVolumeProperty:
+newOmRequest = handleSetVolumePropertyStart(omRequest);
+break;
+ case DeleteVolume:
+newOmRequest = handleDeleteVolumeStart(omRequest);
+break;
+ default:
+new OMException("Unrecognized Command Type:" + cmdType,
+OMException.ResultCodes.INVALID_REQUEST);
+ }
+} catch (IOException ex) {
+ throw ex;
+}
+return newOmRequest;
+ }
+
+
+ @Override
+ public OMResponse handleApplyTransaction(OMRequest omRequest) {
+LOG.debug("Received OMRequest: {}, ", omRequest);
+Type cmdType = omRequest.getCmdType();
+OMResponse.Builder responseBuilder = OMResponse.newBuilder()
+.setCmdType(cmdType)
+.setStatus(Status.OK);
+try {
+ switch (cmdType) {
+ case CreateVolume:
+responseBuilder.setCreateVolumeResponse(
+handleCreateVolumeApply(omRequest));
+break;
+ case SetVolumeProperty:
+responseBuilder.setSetVolumePropertyResponse(
+handleSetVolumePropertyApply(omRequest));
+break;
+ case DeleteVolume:
+responseBuilder.setDeleteVolumeResponse(
+handleDeleteVolumeApply(omRequest));
+break;
+ default:
+// As all request types are not changed so we need to call handle
+// here.
+return handle(omRequest);
+ }
+ responseBuilder.setSuccess(true);
+} catch (IOException ex) {
+ responseBuilder.setSuccess(false);
+ responseBuilder.setStatus(exceptionToResponseStatus(ex));
+ if (ex.getMessage() != null) {
+responseBuilder.setMessage(ex.getMessage());
+ }
+}
+return responseBuilder.build();
+ }
+
+
+ private OMRequest handleCreateVolumeStart(OMRequest omRequest)
+ throws IOException {
+try {
+ OzoneManagerProtocolProtos.VolumeInfo volumeInfo =
+ omRequest.getCreateVolumeRequest().getVolumeInfo();
+ OzoneManagerProtocolProtos.VolumeList volumeList =
+ impl.startCreateVolume(OmVolumeArgs.getFromProtobuf(volumeInfo));
+
+ CreateVolumeRequest createVolumeRequest =
+ CreateVolumeRequest.newBuilder().setVolumeInfo(volumeInfo)
+ .setVolumeList(volumeList).build();
+ return omRequest.toBuilder().setCreateVolumeRequest(createVolumeRequest)
+ .build();
+} catch (IOException ex) {
+ throw ex;
+}
+ }
+
+ private CreateVolumeResponse handleCreateVolumeApply(OMRequest omRequest)
+ throws IOException {
+try {
+ OzoneManagerProtocolProtos.VolumeInfo volumeInfo =
+ omRequest.getCreateVolumeRequest().getVolumeInfo();
+ OzoneManagerProtocolProtos.VolumeList volumeList =
+ omRequest.getCreateVolumeRequest().getVolumeList();
+ impl.applyCreateVolume(OmVolumeArgs.getFromProtobuf(volumeInfo),
+ volumeList);
+} catch (IOException ex) {
+ throw ex;
+}
+return CreateVolumeResponse.newBuilder().build();
+ }
+
+ private OMRequest handleSetVolumePropertyStart(OMRequest omRequest)
+ throws IOException {
+SetVolumePropertyRequest setVolumePropertyRequest =
+omRequest.getSetVolumePropertyRequest();
+String volume = setVolumePropertyRequest.getVolumeName();
+OMRequest newOmRequest = null;
+if (setVolumePropertyRequest.hasQuotaInBytes()) {
+ long quota = setVolumePropertyRequest.getQuotaInBytes();
+ OmVolumeArgs omVolumeArgs = impl.startSetQuota(volume, quota);
+ SetVolumePropertyRequest newSetVolumePropertyRequest =
+ SetVolumePropertyRequest.newBuilder().setVolumeName(volume)
+ .setVolumeInfo(omVolumeArgs.getProtobuf()).build();
+ newOmRequest =
+
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222709=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222709
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 03/Apr/19 23:55
Start Date: 03/Apr/19 23:55
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #689:
HDDS-1379. Convert all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271975546
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/VolumeManagerImpl.java
##
@@ -322,28 +409,56 @@ public void deleteVolume(String volume) throws
IOException {
Preconditions.checkState(volume.equals(volumeArgs.getVolume()));
// delete the volume from the owner list
// as well as delete the volume entry
- try (BatchOperation batch = metadataManager.getStore()
- .initBatchOperation()) {
-delVolumeFromOwnerList(volume, volumeArgs.getOwnerName(), batch);
-metadataManager.getVolumeTable().deleteWithBatch(batch, dbVolumeKey);
-metadataManager.getStore().commitBatchOperation(batch);
+ VolumeList newVolumeList = delVolumeFromOwnerList(volume,
+ volumeArgs.getOwnerName());
+
+ if (!isRatisEnabled) {
+deleteVolumeCommitToDB(newVolumeList,
+volume, owner);
}
-} catch (RocksDBException| IOException ex) {
+ return new OmDeleteVolumeResponse(volume, owner, newVolumeList);
+} catch (IOException ex) {
if (!(ex instanceof OMException)) {
LOG.error("Delete volume failed for volume:{}", volume, ex);
}
- if(ex instanceof RocksDBException) {
-throw RocksDBStore.toIOException("Volume creation failed.",
-(RocksDBException) ex);
- } else {
-throw (IOException) ex;
- }
+ throw ex;
} finally {
metadataManager.getLock().releaseVolumeLock(volume);
metadataManager.getLock().releaseUserLock(owner);
}
}
+ @Override
+ public void applyDeleteVolume(String volume, String owner,
+ VolumeList newVolumeList) throws IOException {
+try {
+ deleteVolumeCommitToDB(newVolumeList, volume, owner);
+} catch (IOException ex) {
+ LOG.error("Delete volume failed for volume:{}", volume,
+ ex);
+ throw ex;
+}
+ }
+
+ private void deleteVolumeCommitToDB(VolumeList newVolumeList,
+ String volume, String owner) throws IOException {
+try (BatchOperation batch = metadataManager.getStore()
+.initBatchOperation()) {
+ String dbUserKey = metadataManager.getUserKey(owner);
Review comment:
This is not DB reading, this method returns the same user name which we have
passed. I think this was added for future purpose I believe if we want to have
a different key for user
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 222709)
Time Spent: 2h 10m (was: 2h)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222708=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222708
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 03/Apr/19 23:55
Start Date: 03/Apr/19 23:55
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #689:
HDDS-1379. Convert all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271975546
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/VolumeManagerImpl.java
##
@@ -322,28 +409,56 @@ public void deleteVolume(String volume) throws
IOException {
Preconditions.checkState(volume.equals(volumeArgs.getVolume()));
// delete the volume from the owner list
// as well as delete the volume entry
- try (BatchOperation batch = metadataManager.getStore()
- .initBatchOperation()) {
-delVolumeFromOwnerList(volume, volumeArgs.getOwnerName(), batch);
-metadataManager.getVolumeTable().deleteWithBatch(batch, dbVolumeKey);
-metadataManager.getStore().commitBatchOperation(batch);
+ VolumeList newVolumeList = delVolumeFromOwnerList(volume,
+ volumeArgs.getOwnerName());
+
+ if (!isRatisEnabled) {
+deleteVolumeCommitToDB(newVolumeList,
+volume, owner);
}
-} catch (RocksDBException| IOException ex) {
+ return new OmDeleteVolumeResponse(volume, owner, newVolumeList);
+} catch (IOException ex) {
if (!(ex instanceof OMException)) {
LOG.error("Delete volume failed for volume:{}", volume, ex);
}
- if(ex instanceof RocksDBException) {
-throw RocksDBStore.toIOException("Volume creation failed.",
-(RocksDBException) ex);
- } else {
-throw (IOException) ex;
- }
+ throw ex;
} finally {
metadataManager.getLock().releaseVolumeLock(volume);
metadataManager.getLock().releaseUserLock(owner);
}
}
+ @Override
+ public void applyDeleteVolume(String volume, String owner,
+ VolumeList newVolumeList) throws IOException {
+try {
+ deleteVolumeCommitToDB(newVolumeList, volume, owner);
+} catch (IOException ex) {
+ LOG.error("Delete volume failed for volume:{}", volume,
+ ex);
+ throw ex;
+}
+ }
+
+ private void deleteVolumeCommitToDB(VolumeList newVolumeList,
+ String volume, String owner) throws IOException {
+try (BatchOperation batch = metadataManager.getStore()
+.initBatchOperation()) {
+ String dbUserKey = metadataManager.getUserKey(owner);
Review comment:
This is not DB reading, this method applys, it returns the same user name
which we have passed. I think this was added for future purpose I believe if we
want to have a different key for user
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 222708)
Time Spent: 2h (was: 1h 50m)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222707=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222707
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 03/Apr/19 23:54
Start Date: 03/Apr/19 23:54
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #689:
HDDS-1379. Convert all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271975546
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/VolumeManagerImpl.java
##
@@ -322,28 +409,56 @@ public void deleteVolume(String volume) throws
IOException {
Preconditions.checkState(volume.equals(volumeArgs.getVolume()));
// delete the volume from the owner list
// as well as delete the volume entry
- try (BatchOperation batch = metadataManager.getStore()
- .initBatchOperation()) {
-delVolumeFromOwnerList(volume, volumeArgs.getOwnerName(), batch);
-metadataManager.getVolumeTable().deleteWithBatch(batch, dbVolumeKey);
-metadataManager.getStore().commitBatchOperation(batch);
+ VolumeList newVolumeList = delVolumeFromOwnerList(volume,
+ volumeArgs.getOwnerName());
+
+ if (!isRatisEnabled) {
+deleteVolumeCommitToDB(newVolumeList,
+volume, owner);
}
-} catch (RocksDBException| IOException ex) {
+ return new OmDeleteVolumeResponse(volume, owner, newVolumeList);
+} catch (IOException ex) {
if (!(ex instanceof OMException)) {
LOG.error("Delete volume failed for volume:{}", volume, ex);
}
- if(ex instanceof RocksDBException) {
-throw RocksDBStore.toIOException("Volume creation failed.",
-(RocksDBException) ex);
- } else {
-throw (IOException) ex;
- }
+ throw ex;
} finally {
metadataManager.getLock().releaseVolumeLock(volume);
metadataManager.getLock().releaseUserLock(owner);
}
}
+ @Override
+ public void applyDeleteVolume(String volume, String owner,
+ VolumeList newVolumeList) throws IOException {
+try {
+ deleteVolumeCommitToDB(newVolumeList, volume, owner);
+} catch (IOException ex) {
+ LOG.error("Delete volume failed for volume:{}", volume,
+ ex);
+ throw ex;
+}
+ }
+
+ private void deleteVolumeCommitToDB(VolumeList newVolumeList,
+ String volume, String owner) throws IOException {
+try (BatchOperation batch = metadataManager.getStore()
+.initBatchOperation()) {
+ String dbUserKey = metadataManager.getUserKey(owner);
Review comment:
This is not DB reading, this method applys "/" before volume.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 222707)
Time Spent: 1h 50m (was: 1h 40m)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222706=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222706
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 03/Apr/19 23:54
Start Date: 03/Apr/19 23:54
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #689:
HDDS-1379. Convert all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271975519
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/VolumeManagerImpl.java
##
@@ -255,6 +328,19 @@ public void setQuota(String volume, long quota) throws
IOException {
}
}
+ @Override
+ public void applySetQuota(OmVolumeArgs omVolumeArgs) throws IOException {
+try {
+ String dbVolumeKey = metadataManager.getVolumeKey(
Review comment:
This is not DB reading, this method applys "/" before volume.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 222706)
Time Spent: 1h 40m (was: 1.5h)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222690=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222690
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 03/Apr/19 23:32
Start Date: 03/Apr/19 23:32
Worklog Time Spent: 10m
Work Description: arp7 commented on pull request #689: HDDS-1379. Convert
all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271970646
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##
@@ -1594,6 +1598,107 @@ public void createVolume(OmVolumeArgs args) throws
IOException {
}
}
+ @Override
+ public VolumeList startCreateVolume(OmVolumeArgs args) throws IOException {
+try {
+ // TODO: Need to add metrics and Audit log for HA requests
+ if(isAclEnabled) {
Review comment:
Nitpick: need space between if and (
This needs to be fixed in multiple places.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 222690)
Time Spent: 1h (was: 50m)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222695=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222695
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 03/Apr/19 23:32
Start Date: 03/Apr/19 23:32
Worklog Time Spent: 10m
Work Description: arp7 commented on pull request #689: HDDS-1379. Convert
all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271971268
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/VolumeManagerImpl.java
##
@@ -322,28 +409,56 @@ public void deleteVolume(String volume) throws
IOException {
Preconditions.checkState(volume.equals(volumeArgs.getVolume()));
// delete the volume from the owner list
// as well as delete the volume entry
- try (BatchOperation batch = metadataManager.getStore()
- .initBatchOperation()) {
-delVolumeFromOwnerList(volume, volumeArgs.getOwnerName(), batch);
-metadataManager.getVolumeTable().deleteWithBatch(batch, dbVolumeKey);
-metadataManager.getStore().commitBatchOperation(batch);
+ VolumeList newVolumeList = delVolumeFromOwnerList(volume,
+ volumeArgs.getOwnerName());
+
+ if (!isRatisEnabled) {
+deleteVolumeCommitToDB(newVolumeList,
+volume, owner);
}
-} catch (RocksDBException| IOException ex) {
+ return new OmDeleteVolumeResponse(volume, owner, newVolumeList);
+} catch (IOException ex) {
if (!(ex instanceof OMException)) {
LOG.error("Delete volume failed for volume:{}", volume, ex);
}
- if(ex instanceof RocksDBException) {
-throw RocksDBStore.toIOException("Volume creation failed.",
-(RocksDBException) ex);
- } else {
-throw (IOException) ex;
- }
+ throw ex;
} finally {
metadataManager.getLock().releaseVolumeLock(volume);
metadataManager.getLock().releaseUserLock(owner);
}
}
+ @Override
+ public void applyDeleteVolume(String volume, String owner,
+ VolumeList newVolumeList) throws IOException {
+try {
+ deleteVolumeCommitToDB(newVolumeList, volume, owner);
+} catch (IOException ex) {
+ LOG.error("Delete volume failed for volume:{}", volume,
+ ex);
+ throw ex;
+}
+ }
+
+ private void deleteVolumeCommitToDB(VolumeList newVolumeList,
+ String volume, String owner) throws IOException {
+try (BatchOperation batch = metadataManager.getStore()
+.initBatchOperation()) {
+ String dbUserKey = metadataManager.getUserKey(owner);
Review comment:
Same. Can we pass the userKey from start to apply?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 222695)
Time Spent: 1.5h (was: 1h 20m)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222693=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222693
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 03/Apr/19 23:32
Start Date: 03/Apr/19 23:32
Worklog Time Spent: 10m
Work Description: arp7 commented on pull request #689: HDDS-1379. Convert
all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271970849
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/S3BucketManagerImpl.java
##
@@ -166,6 +170,10 @@ public boolean createOzoneVolumeIfNeeded(String userName)
.setVolume(ozoneVolumeName)
.setQuotaInBytes(OzoneConsts.MAX_QUOTA_IN_BYTES)
.build();
+ if (isRatisEnabled) {
Review comment:
Why do we need to call applyCreateVolume separately? Is this being pushed
through Ratis consensus correctly?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 222693)
Time Spent: 1h 20m (was: 1h 10m)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222692=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222692
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 03/Apr/19 23:32
Start Date: 03/Apr/19 23:32
Worklog Time Spent: 10m
Work Description: arp7 commented on pull request #689: HDDS-1379. Convert
all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271971359
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/OzoneManagerRequestHandler.java
##
@@ -363,6 +365,199 @@ public OMResponse handle(OMRequest request) {
return responseBuilder.build();
}
+ @Override
+ public OMRequest handleStartTransaction(OMRequest omRequest)
+ throws IOException {
+LOG.debug("Received OMRequest: {}, ", omRequest);
+Type cmdType = omRequest.getCmdType();
+OMRequest newOmRequest = null;
+try {
+ switch (cmdType) {
+ case CreateVolume:
+newOmRequest = handleCreateVolumeStart(omRequest);
+break;
+ case SetVolumeProperty:
+newOmRequest = handleSetVolumePropertyStart(omRequest);
+break;
+ case DeleteVolume:
+newOmRequest = handleDeleteVolumeStart(omRequest);
+break;
+ default:
+new OMException("Unrecognized Command Type:" + cmdType,
+OMException.ResultCodes.INVALID_REQUEST);
+ }
+} catch (IOException ex) {
+ throw ex;
+}
+return newOmRequest;
+ }
+
+
+ @Override
+ public OMResponse handleApplyTransaction(OMRequest omRequest) {
+LOG.debug("Received OMRequest: {}, ", omRequest);
+Type cmdType = omRequest.getCmdType();
+OMResponse.Builder responseBuilder = OMResponse.newBuilder()
+.setCmdType(cmdType)
+.setStatus(Status.OK);
+try {
+ switch (cmdType) {
+ case CreateVolume:
+responseBuilder.setCreateVolumeResponse(
+handleCreateVolumeApply(omRequest));
+break;
+ case SetVolumeProperty:
+responseBuilder.setSetVolumePropertyResponse(
+handleSetVolumePropertyApply(omRequest));
+break;
+ case DeleteVolume:
+responseBuilder.setDeleteVolumeResponse(
+handleDeleteVolumeApply(omRequest));
+break;
+ default:
+// As all request types are not changed so we need to call handle
+// here.
+return handle(omRequest);
+ }
+ responseBuilder.setSuccess(true);
+} catch (IOException ex) {
+ responseBuilder.setSuccess(false);
+ responseBuilder.setStatus(exceptionToResponseStatus(ex));
+ if (ex.getMessage() != null) {
+responseBuilder.setMessage(ex.getMessage());
+ }
+}
+return responseBuilder.build();
+ }
+
+
+ private OMRequest handleCreateVolumeStart(OMRequest omRequest)
+ throws IOException {
+try {
+ OzoneManagerProtocolProtos.VolumeInfo volumeInfo =
+ omRequest.getCreateVolumeRequest().getVolumeInfo();
+ OzoneManagerProtocolProtos.VolumeList volumeList =
+ impl.startCreateVolume(OmVolumeArgs.getFromProtobuf(volumeInfo));
+
+ CreateVolumeRequest createVolumeRequest =
+ CreateVolumeRequest.newBuilder().setVolumeInfo(volumeInfo)
+ .setVolumeList(volumeList).build();
+ return omRequest.toBuilder().setCreateVolumeRequest(createVolumeRequest)
+ .build();
+} catch (IOException ex) {
+ throw ex;
+}
+ }
+
+ private CreateVolumeResponse handleCreateVolumeApply(OMRequest omRequest)
+ throws IOException {
+try {
+ OzoneManagerProtocolProtos.VolumeInfo volumeInfo =
+ omRequest.getCreateVolumeRequest().getVolumeInfo();
+ OzoneManagerProtocolProtos.VolumeList volumeList =
+ omRequest.getCreateVolumeRequest().getVolumeList();
+ impl.applyCreateVolume(OmVolumeArgs.getFromProtobuf(volumeInfo),
+ volumeList);
+} catch (IOException ex) {
+ throw ex;
+}
+return CreateVolumeResponse.newBuilder().build();
+ }
+
+ private OMRequest handleSetVolumePropertyStart(OMRequest omRequest)
+ throws IOException {
+SetVolumePropertyRequest setVolumePropertyRequest =
+omRequest.getSetVolumePropertyRequest();
+String volume = setVolumePropertyRequest.getVolumeName();
+OMRequest newOmRequest = null;
+if (setVolumePropertyRequest.hasQuotaInBytes()) {
+ long quota = setVolumePropertyRequest.getQuotaInBytes();
+ OmVolumeArgs omVolumeArgs = impl.startSetQuota(volume, quota);
+ SetVolumePropertyRequest newSetVolumePropertyRequest =
+ SetVolumePropertyRequest.newBuilder().setVolumeName(volume)
+ .setVolumeInfo(omVolumeArgs.getProtobuf()).build();
+ newOmRequest =
+
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222691=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222691
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 03/Apr/19 23:32
Start Date: 03/Apr/19 23:32
Worklog Time Spent: 10m
Work Description: arp7 commented on pull request #689: HDDS-1379. Convert
all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271970765
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##
@@ -1594,6 +1598,107 @@ public void createVolume(OmVolumeArgs args) throws
IOException {
}
}
+ @Override
+ public VolumeList startCreateVolume(OmVolumeArgs args) throws IOException {
+try {
+ // TODO: Need to add metrics and Audit log for HA requests
+ if(isAclEnabled) {
+checkAcls(ResourceType.VOLUME, StoreType.OZONE,
+ACLType.CREATE, args.getVolume(), null, null);
+ }
+ VolumeList volumeList = volumeManager.createVolume(args);
+ return volumeList;
+} catch (Exception ex) {
+ throw ex;
Review comment:
Don't catch and throw same exception. This is an anti-pattern. This needs to
be fixed in multiple places in different files.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 222691)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222689=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222689
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 03/Apr/19 23:32
Start Date: 03/Apr/19 23:32
Worklog Time Spent: 10m
Work Description: arp7 commented on pull request #689: HDDS-1379. Convert
all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271970445
##
File path:
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmVolumeOwnerChangeResponse.java
##
@@ -0,0 +1,56 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.helpers;
+
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos
+.VolumeList;
+
+/**
+ * OM response for owner change request for a ozone volume.
+ */
+public class OmVolumeOwnerChangeResponse {
+ private VolumeList originalOwnerVolumeList;
Review comment:
Why does the response need to have all these fields? Still reviewing so it
may be clearer later.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 222689)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222694=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222694
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 03/Apr/19 23:32
Start Date: 03/Apr/19 23:32
Worklog Time Spent: 10m
Work Description: arp7 commented on pull request #689: HDDS-1379. Convert
all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271971191
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/VolumeManagerImpl.java
##
@@ -255,6 +328,19 @@ public void setQuota(String volume, long quota) throws
IOException {
}
}
+ @Override
+ public void applySetQuota(OmVolumeArgs omVolumeArgs) throws IOException {
+try {
+ String dbVolumeKey = metadataManager.getVolumeKey(
Review comment:
We should not need to do a read in applySetQuota. Can we pass the
dbVolumeKey from start to apply?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 222694)
Time Spent: 1h 20m (was: 1h 10m)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[
https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222688=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222688
]
ASF GitHub Bot logged work on HDDS-1379:
Author: ASF GitHub Bot
Created on: 03/Apr/19 23:32
Start Date: 03/Apr/19 23:32
Worklog Time Spent: 10m
Work Description: arp7 commented on pull request #689: HDDS-1379. Convert
all OM Volume related operations to HA model.
URL: https://github.com/apache/hadoop/pull/689#discussion_r271970568
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
##
@@ -1594,6 +1598,107 @@ public void createVolume(OmVolumeArgs args) throws
IOException {
}
}
+ @Override
+ public VolumeList startCreateVolume(OmVolumeArgs args) throws IOException {
+try {
+ // TODO: Need to add metrics and Audit log for HA requests
Review comment:
Let's file a follow up jira. Audit especially is critical to fix.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 222688)
Time Spent: 50m (was: 40m)
> Convert all OM Volume related operations to HA model
>
>
> Key: HDDS-1379
> URL: https://issues.apache.org/jira/browse/HDDS-1379
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> In this jira, we shall convert all OM related operations to OM HA model,
> which is a 2 step.
> # StartTransaction, where we validate request and check for any errors and
> return the response.
> # ApplyTransaction, where original OM request will have a response which
> needs to be applied to OM DB. This step is just to apply response to Om DB.
> In this way, all requests which are failed with like volume not found or some
> conditions which i have not satisfied like when deleting volume should be
> empty, these all will be executed during startTransaction, and if it fails
> these requests will not be written to raft log also.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[ https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222606=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222606 ] ASF GitHub Bot logged work on HDDS-1379: Author: ASF GitHub Bot Created on: 03/Apr/19 20:30 Start Date: 03/Apr/19 20:30 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #689: HDDS-1379. Convert all OM Volume related operations to HA model. URL: https://github.com/apache/hadoop/pull/689#issuecomment-479647097 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 26 | Docker mode activated. | ||| _ Prechecks _ | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 63 | Maven dependency ordering for branch | | +1 | mvninstall | 1034 | trunk passed | | +1 | compile | 98 | trunk passed | | +1 | checkstyle | 26 | trunk passed | | +1 | mvnsite | 97 | trunk passed | | +1 | shadedclient | 750 | branch has no errors when building and testing our client artifacts. | | 0 | findbugs | 0 | Skipped patched modules with no Java source: hadoop-ozone/integration-test | | +1 | findbugs | 93 | trunk passed | | +1 | javadoc | 68 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 10 | Maven dependency ordering for patch | | +1 | mvninstall | 97 | the patch passed | | +1 | compile | 88 | the patch passed | | +1 | cc | 88 | the patch passed | | +1 | javac | 88 | the patch passed | | -0 | checkstyle | 22 | hadoop-ozone: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | +1 | mvnsite | 79 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 706 | patch has no errors when building and testing our client artifacts. | | 0 | findbugs | 0 | Skipped patched modules with no Java source: hadoop-ozone/integration-test | | -1 | findbugs | 50 | hadoop-ozone/ozone-manager generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | -1 | javadoc | 35 | hadoop-ozone_common generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | ||| _ Other Tests _ | | +1 | unit | 36 | common in the patch passed. | | +1 | unit | 42 | ozone-manager in the patch passed. | | -1 | unit | 784 | integration-test in the patch failed. | | +1 | asflicense | 34 | The patch does not generate ASF License warnings. | | | | 4380 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-ozone/ozone-manager | | | Dead store to volume in org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleSetVolumePropertyApply(OzoneManagerProtocolProtos$OMRequest) At OzoneManagerRequestHandler.java:org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleSetVolumePropertyApply(OzoneManagerProtocolProtos$OMRequest) At OzoneManagerRequestHandler.java:[line 512] | | | new org.apache.hadoop.ozone.om.exceptions.OMException(String, OMException$ResultCodes) not thrown in org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleStartTransaction(OzoneManagerProtocolProtos$OMRequest) At OzoneManagerRequestHandler.java:in org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleStartTransaction(OzoneManagerProtocolProtos$OMRequest) At OzoneManagerRequestHandler.java:[line 386] | | Failed junit tests | hadoop.ozone.om.TestOmMetrics | | | hadoop.ozone.scm.pipeline.TestPipelineManagerMXBean | | | hadoop.ozone.TestMiniChaosOzoneCluster | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-689/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/689 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux 2fe446930f83 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / d797907 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_191 | | findbugs | v3.1.0-RC1 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-689/1/artifact/out/diff-checkstyle-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-689/1/artifact/out/new-findbugs-hadoop-ozone_ozone-manager.html | | javadoc |
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[ https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222586=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222586 ] ASF GitHub Bot logged work on HDDS-1379: Author: ASF GitHub Bot Created on: 03/Apr/19 19:43 Start Date: 03/Apr/19 19:43 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #688: HDDS-1379. Convert all OM Volume related operations to HA model URL: https://github.com/apache/hadoop/pull/688#issuecomment-479631355 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 25 | Docker mode activated. | ||| _ Prechecks _ | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 59 | Maven dependency ordering for branch | | +1 | mvninstall | 1028 | trunk passed | | +1 | compile | 99 | trunk passed | | +1 | checkstyle | 29 | trunk passed | | +1 | mvnsite | 113 | trunk passed | | +1 | shadedclient | 813 | branch has no errors when building and testing our client artifacts. | | 0 | findbugs | 0 | Skipped patched modules with no Java source: hadoop-ozone/integration-test | | +1 | findbugs | 102 | trunk passed | | +1 | javadoc | 84 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 13 | Maven dependency ordering for patch | | +1 | mvninstall | 104 | the patch passed | | +1 | compile | 92 | the patch passed | | +1 | cc | 92 | the patch passed | | +1 | javac | 92 | the patch passed | | -0 | checkstyle | 24 | hadoop-ozone: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | +1 | mvnsite | 87 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 744 | patch has no errors when building and testing our client artifacts. | | 0 | findbugs | 0 | Skipped patched modules with no Java source: hadoop-ozone/integration-test | | -1 | findbugs | 46 | hadoop-ozone/ozone-manager generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | -1 | javadoc | 33 | hadoop-ozone_common generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | ||| _ Other Tests _ | | +1 | unit | 32 | common in the patch passed. | | +1 | unit | 39 | ozone-manager in the patch passed. | | -1 | unit | 684 | integration-test in the patch failed. | | +1 | asflicense | 31 | The patch does not generate ASF License warnings. | | | | 4409 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-ozone/ozone-manager | | | Dead store to volume in org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleSetVolumePropertyApply(OzoneManagerProtocolProtos$OMRequest) At OzoneManagerRequestHandler.java:org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleSetVolumePropertyApply(OzoneManagerProtocolProtos$OMRequest) At OzoneManagerRequestHandler.java:[line 512] | | | new org.apache.hadoop.ozone.om.exceptions.OMException(String, OMException$ResultCodes) not thrown in org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleStartTransaction(OzoneManagerProtocolProtos$OMRequest) At OzoneManagerRequestHandler.java:in org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleStartTransaction(OzoneManagerProtocolProtos$OMRequest) At OzoneManagerRequestHandler.java:[line 386] | | Failed junit tests | hadoop.ozone.om.TestOmMetrics | | | hadoop.ozone.TestMiniChaosOzoneCluster | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-688/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/688 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux 734c03b03799 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / be488b6 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_191 | | findbugs | v3.1.0-RC1 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-688/1/artifact/out/diff-checkstyle-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-688/1/artifact/out/new-findbugs-hadoop-ozone_ozone-manager.html | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-688/1/artifact/out/diff-javadoc-javadoc-hadoop-ozone_common.txt | | unit |
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[ https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222561=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222561 ] ASF GitHub Bot logged work on HDDS-1379: Author: ASF GitHub Bot Created on: 03/Apr/19 19:16 Start Date: 03/Apr/19 19:16 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on pull request #689: HDDS-1379. Convert all OM Volume related operations to HA model. URL: https://github.com/apache/hadoop/pull/689 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 222561) Time Spent: 20m (was: 10m) > Convert all OM Volume related operations to HA model > > > Key: HDDS-1379 > URL: https://issues.apache.org/jira/browse/HDDS-1379 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > In this jira, we shall convert all OM related operations to OM HA model, > which is a 2 step. > # StartTransaction, where we validate request and check for any errors and > return the response. > # ApplyTransaction, where original OM request will have a response which > needs to be applied to OM DB. This step is just to apply response to Om DB. > In this way, all requests which are failed with like volume not found or some > conditions which i have not satisfied like when deleting volume should be > empty, these all will be executed during startTransaction, and if it fails > these requests will not be written to raft log also. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1379) Convert all OM Volume related operations to HA model
[ https://issues.apache.org/jira/browse/HDDS-1379?focusedWorklogId=222559=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-222559 ] ASF GitHub Bot logged work on HDDS-1379: Author: ASF GitHub Bot Created on: 03/Apr/19 19:12 Start Date: 03/Apr/19 19:12 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on pull request #688: HDDS-1379. Convert all OM Volume related operations to HA model URL: https://github.com/apache/hadoop/pull/688 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 222559) Time Spent: 10m Remaining Estimate: 0h > Convert all OM Volume related operations to HA model > > > Key: HDDS-1379 > URL: https://issues.apache.org/jira/browse/HDDS-1379 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > In this jira, we shall convert all OM related operations to OM HA model, > which is a 2 step. > # StartTransaction, where we validate request and check for any errors and > return the response. > # ApplyTransaction, where original OM request will have a response which > needs to be applied to OM DB. This step is just to apply response to Om DB. > In this way, all requests which are failed with like volume not found or some > conditions which i have not satisfied like when deleting volume should be > empty, these all will be executed during startTransaction, and if it fails > these requests will not be written to raft log also. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
