[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14026555#comment-14026555
]
Hudson commented on YARN-2030:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1797 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1797/])
YARN-2030. Augmented RMStateStore with state machine. Contributed by Binglin
Chang (jianhe:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1601537)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Fix For: 2.5.0
>
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch,
> YARN-2030.v3.patch, YARN-2030.v4.patch, YARN-2030.v5.patch, YARN-2030.v6.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14026453#comment-14026453
]
Hudson commented on YARN-2030:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #1770 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1770/])
YARN-2030. Augmented RMStateStore with state machine. Contributed by Binglin
Chang (jianhe:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1601537)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Fix For: 2.5.0
>
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch,
> YARN-2030.v3.patch, YARN-2030.v4.patch, YARN-2030.v5.patch, YARN-2030.v6.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14026336#comment-14026336
]
Hudson commented on YARN-2030:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #579 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/579/])
YARN-2030. Augmented RMStateStore with state machine. Contributed by Binglin
Chang (jianhe:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1601537)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Fix For: 2.5.0
>
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch,
> YARN-2030.v3.patch, YARN-2030.v4.patch, YARN-2030.v5.patch, YARN-2030.v6.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14026005#comment-14026005
]
Hudson commented on YARN-2030:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #5672 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/5672/])
YARN-2030. Augmented RMStateStore with state machine. Contributed by Binglin
Chang (jianhe:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1601537)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Fix For: 2.5.0
>
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch,
> YARN-2030.v3.patch, YARN-2030.v4.patch, YARN-2030.v5.patch, YARN-2030.v6.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025648#comment-14025648
]
Jian He commented on YARN-2030:
---
Opened YARN-2138 for handling un-handled store op failed exceptions.
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Fix For: 2.5.0
>
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch,
> YARN-2030.v3.patch, YARN-2030.v4.patch, YARN-2030.v5.patch, YARN-2030.v6.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025626#comment-14025626
]
Jian He commented on YARN-2030:
---
Noticed an existing issue that the storedException passed into
notifyDoneStoringApplication is always null and
RMFatalEventType.STATE_STORE_OP_FAILED is not handled anywhere, we can fix this
separately.
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch,
> YARN-2030.v3.patch, YARN-2030.v4.patch, YARN-2030.v5.patch, YARN-2030.v6.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025599#comment-14025599
]
Jian He commented on YARN-2030:
---
Committing this. Filed YARN-2136 for handling events at fenced state.
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch,
> YARN-2030.v3.patch, YARN-2030.v4.patch, YARN-2030.v5.patch, YARN-2030.v6.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025501#comment-14025501
]
Hadoop QA commented on YARN-2030:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12649401/YARN-2030.v6.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3945//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3945//console
This message is automatically generated.
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch,
> YARN-2030.v3.patch, YARN-2030.v4.patch, YARN-2030.v5.patch, YARN-2030.v6.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025467#comment-14025467
]
Jian He commented on YARN-2030:
---
bq. we should perhaps have more states like for fencing.
Makes sense, it's useful to have a fenced state to handle events upfront
instead of invoking more zk operations
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch,
> YARN-2030.v3.patch, YARN-2030.v4.patch, YARN-2030.v5.patch, YARN-2030.v6.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025451#comment-14025451
]
Vinod Kumar Vavilapalli commented on YARN-2030:
---
Quickly looked at it. Fine overall.
So, the store itself is a state machine now with just one state. It'd have been
better if there were more states. We should perhaps have more states like for
fencing. I think even today the store may handle some events, even though the
store is in the process of getting fenced/failed when one of the
RMFatalEventType happens. State machine is precisely useful for unearthing
issues like this. Anyways that's for future.
The code does look a little simpler now. This patch looks okay to me too.
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch,
> YARN-2030.v3.patch, YARN-2030.v4.patch, YARN-2030.v5.patch, YARN-2030.v6.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025421#comment-14025421
]
Jian He commented on YARN-2030:
---
Patch looks good to me, did a minor rename from operand -> store myself
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch,
> YARN-2030.v3.patch, YARN-2030.v4.patch, YARN-2030.v5.patch, YARN-2030.v6.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14021156#comment-14021156
]
Hadoop QA commented on YARN-2030:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12648857/YARN-2030.v5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3938//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3938//console
This message is automatically generated.
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch,
> YARN-2030.v3.patch, YARN-2030.v4.patch, YARN-2030.v5.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14020741#comment-14020741
]
Hadoop QA commented on YARN-2030:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12648801/YARN-2030.v4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3937//console
This message is automatically generated.
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch,
> YARN-2030.v3.patch, YARN-2030.v4.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14019588#comment-14019588
]
Jian He commented on YARN-2030:
---
I meant add an abstract getProto method.
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch,
> YARN-2030.v3.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14019581#comment-14019581
]
Jian He commented on YARN-2030:
---
I see, thanks for the update. Maybe just promote getProto() to the abstract
class, given this record is used internally by RM only? should be fine?
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch,
> YARN-2030.v3.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14018758#comment-14018758
]
Junping Du commented on YARN-2030:
--
bq. Junping Du, can you help with the review and commit ? thx.
Sure. I am glad to help here.
[~decster], we want to have abstract class here as we want to provide as simple
interface as possible to end user (or AM) who can just simply call
ApplicationStateData.newInstance() without involving the complexity of
ApplicationStateDataPBImpl. Make sense?
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14018478#comment-14018478
]
Jian He commented on YARN-2030:
---
bq. looks like PBImpl already has ProtoBase as super class, so we can't change
interface to abstract class
We can merge the stuff from ProtoBase into the pb class and get rid of the
ProtoBase, as was done for other user-facing records.
[~djp], can you help with the review and commit ? thx.
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14018335#comment-14018335
]
Junping Du commented on YARN-2030:
--
[~decster], I think He Jian was reviewing your patch.
[~jianhe], I know you are quite busy on Hadoop Summit recently. Do you mind to
review this patch again after that or you want me to review it?
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016325#comment-14016325
]
Binglin Chang commented on YARN-2030:
-
Hi, [~djp]. I think the patch is ready for review, could you help review the
patch?
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14007919#comment-14007919
]
Hadoop QA commented on YARN-2030:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12645932/YARN-2030.v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3817//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3817//console
This message is automatically generated.
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006858#comment-14006858
]
Hadoop QA commented on YARN-2030:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12645932/YARN-2030.v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3793//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3793//console
This message is automatically generated.
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14005753#comment-14005753
]
Binglin Chang commented on YARN-2030:
-
Hi Jian He,
Thanks for the comments, looks like PBImpl already has ProtoBase as super
class, so we can't change interface to abstract class
{code}
public class ApplicationAttemptStateDataPBImpl
extends ProtoBase
implements ApplicationAttemptStateData {
{code}
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14004398#comment-14004398
]
Jian He commented on YARN-2030:
---
Hi Binglin, thanks for cleaning it up !
Just quickly looked at the patch. some suggestion. I think ApplicationStateData
can be changed to an abstract class just like many other records and add the
newInstance() method. So things like ApplicationStateDataPBImpl don't need the
newApplicationStateData method. Accordingly storeApplicationStateInternal can
take in ApplicationStateData instead of ApplicationStateDataPBImpl as the
argument to avoid the type cast.
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14004335#comment-14004335
]
Junping Du commented on YARN-2030:
--
Hi [~decster], thanks for taking on this effort. I will review your patch.
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch, YARN-2030.v2.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14003208#comment-14003208
]
Hadoop QA commented on YARN-2030:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12645754/YARN-2030.v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3772//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/3772//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3772//console
This message is automatically generated.
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>Assignee: Binglin Chang
> Attachments: YARN-2030.v1.patch
>
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13998635#comment-13998635
]
Tsuyoshi OZAWA commented on YARN-2030:
--
+1 for this idea too. Using state machine makes the code cleaner and simpler.
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2030) Use StateMachine to simplify handleStoreEvent() in RMStateStore
[
https://issues.apache.org/jira/browse/YARN-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993246#comment-13993246
]
Wangda Tan commented on YARN-2030:
--
+1 for this idea, I think we should handle this neatly to avoid possible bugs.
> Use StateMachine to simplify handleStoreEvent() in RMStateStore
> ---
>
> Key: YARN-2030
> URL: https://issues.apache.org/jira/browse/YARN-2030
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Junping Du
>
> Now the logic to handle different store events in handleStoreEvent() is as
> following:
> {code}
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> ...
> try {
> if (event.getType().equals(RMStateStoreEventType.STORE_APP)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)
> || event.getType().equals(RMStateStoreEventType.UPDATE_APP_ATTEMPT)) {
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> ...
> if (event.getType().equals(RMStateStoreEventType.STORE_APP_ATTEMPT)) {
> ...
> } else {
> ...
> }
> }
> ...
> } else if (event.getType().equals(RMStateStoreEventType.REMOVE_APP)) {
> ...
> } else {
> ...
> }
> }
> {code}
> This is not only confuse people but also led to mistake easily. We may
> leverage state machine to simply this even no state transitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
