[jira] [Updated] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[
https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated YARN-2716:
---
Attachment: yarn-2716-2.patch
New patch to address review comments.
[~jianhe] - once you are comfortable with the patch, I would like to move
methods in {{ZKRMStateStore}} around to put all Curator-related methods
together towards the end for better readability.
Refactor ZKRMStateStore retry code with Apache Curator
--
Key: YARN-2716
URL: https://issues.apache.org/jira/browse/YARN-2716
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Jian He
Assignee: Karthik Kambatla
Attachments: yarn-2716-1.patch, yarn-2716-2.patch,
yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch
Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to
simplify the retry logic in ZKRMStateStore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-2716: --- Attachment: yarn-2716-3.patch New patch just moves the class members (fields and methods) around to put all curator-related methods together. Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Karthik Kambatla Attachments: yarn-2716-1.patch, yarn-2716-2.patch, yarn-2716-3.patch, yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-2716: --- Issue Type: Improvement (was: Bug) Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Improvement Reporter: Jian He Assignee: Karthik Kambatla Attachments: yarn-2716-1.patch, yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-2716: --- Attachment: yarn-2716-1.patch Fixed TestZKRMStateStoreZKClientConnections as well. I believe this patch the v1 patch could some more eyes. Appreciate any feedback. Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Karthik Kambatla Attachments: yarn-2716-1.patch, yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-2716: --- Attachment: yarn-2716-prelim.patch I was able to make more progress here. Here is an updated patch (preliminary): # Got rid of ZK.multi in favor of Curator # Most of the tests in TestZKRMStateStore are fixed now, one more fix to go. Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Karthik Kambatla Attachments: yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-2716: --- Attachment: yarn-2716-prelim.patch Updated prelim patch to fix TestZKRMStateStore failures. TestZKRMStateStoreZKClientConnections is yet to be fixed. Some of the tests there check if ZooKeeper operations succeed. Given we are using Curator, I am not sure if all the tests are required. Will have to look at them one-by-one. Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Karthik Kambatla Attachments: yarn-2716-prelim.patch, yarn-2716-prelim.patch, yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2716) Refactor ZKRMStateStore retry code with Apache Curator
[ https://issues.apache.org/jira/browse/YARN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-2716: --- Attachment: yarn-2716-super-prelim.patch Had some time today to look at this again. Here is a super-preliminary patch, in case I fail to spend more time and wrap this up. Pending items: # Use Curator transactions to handle multi. # Fix the test failures in TestZKRMStateStore. I tried catching NoNode and NodeExists in delete and create, but that led to protobuf issues. # Rewrite TestZKRMStateStoreZKClientConnections Refactor ZKRMStateStore retry code with Apache Curator -- Key: YARN-2716 URL: https://issues.apache.org/jira/browse/YARN-2716 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Karthik Kambatla Attachments: yarn-2716-super-prelim.patch Per suggestion by [~kasha] in YARN-2131, it's nice to use curator to simplify the retry logic in ZKRMStateStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
