[jira] [Commented] (NIFI-5204) When node joins cluster, if a processor is stopping but cluster says the state is disabled, node ends up in inconsistent state

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479447#comment-16479447
 ] 

ASF GitHub Bot commented on NIFI-5204:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2713


> When node joins cluster, if a processor is stopping but cluster says the 
> state is disabled, node ends up in inconsistent state
> --
>
> Key: NIFI-5204
> URL: https://issues.apache.org/jira/browse/NIFI-5204
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Critical
> Fix For: 1.7.0
>
>
> In order to make this "easy" to replicate, I did the following:
> 1) Create a 2-node cluster.
>  2) On both nodes, update nifi.properties to set 
> "nifi.variable.registry.properties" to "1.properties"
>  3) On both nodes, create 1.properties in $NIFI_HOME. For first node, set 
> "sleep=2 mins" and for second node, set "sleep=0 millis"
>  4) Update DebugFlow to support expression language for the "@OnStopped Pause 
> Time"
>  5) Configure flow with a DebugFlow processor. Can auto-terminate 
> relationships and set run period to "10 secs."Set "@OnStopped Pause time" to 
> "${sleep}"
>  6) Disable DebugFlow processor.
>  7) Disconnected Node 1.
>  8) Go to Node 1 in browser and Start DebugFlow.
>  9) Stop DebugFlow.
>  10) While processor is still "stopping", go back Node 2 in browser and 
> request that Node 1 re-join the cluster.
> Now, when Node 1 re-joins the cluster, it will attempt to disable the 
> processor but won't be able to because the processor is still stopping. The 
> following will be in the logs:
> {code:java}
> 2018-05-16 15:21:50,986 WARN [Reconnect to Cluster] 
> org.apache.nifi.controller.ProcessorNode Processor cannot be disabled because 
> its state is set to STOPPING{code}
> So we now have a node in an inconsistent state.
> Additionally, if we now go to Node 1 in our browser and unselect all 
> components, and attempt to STOP the process group, the request that is 
> replicated attempts to stop the DebugFlow processor. Node 2 will now fail to 
> stop the processor because the processor is disabled. As a result, Node 2 
> will now be kicked out of the cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5204) When node joins cluster, if a processor is stopping but cluster says the state is disabled, node ends up in inconsistent state

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479443#comment-16479443
 ] 

ASF subversion and git services commented on NIFI-5204:
---

Commit 2afbf963812bccdec8c997e6c63a93c644016769 in nifi's branch 
refs/heads/master from [~markap14]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2afbf96 ]

NIFI-5204: Ensure that verifyCanStop throws ISE if component is disabled
NIFI-5204: If processor joins cluster and inherits 'disabled' state but is 
still stopping, ensure that the state becomes disabled when the processor 
finishes stopping
This closes #2713


> When node joins cluster, if a processor is stopping but cluster says the 
> state is disabled, node ends up in inconsistent state
> --
>
> Key: NIFI-5204
> URL: https://issues.apache.org/jira/browse/NIFI-5204
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Critical
>
> In order to make this "easy" to replicate, I did the following:
> 1) Create a 2-node cluster.
>  2) On both nodes, update nifi.properties to set 
> "nifi.variable.registry.properties" to "1.properties"
>  3) On both nodes, create 1.properties in $NIFI_HOME. For first node, set 
> "sleep=2 mins" and for second node, set "sleep=0 millis"
>  4) Update DebugFlow to support expression language for the "@OnStopped Pause 
> Time"
>  5) Configure flow with a DebugFlow processor. Can auto-terminate 
> relationships and set run period to "10 secs."Set "@OnStopped Pause time" to 
> "${sleep}"
>  6) Disable DebugFlow processor.
>  7) Disconnected Node 1.
>  8) Go to Node 1 in browser and Start DebugFlow.
>  9) Stop DebugFlow.
>  10) While processor is still "stopping", go back Node 2 in browser and 
> request that Node 1 re-join the cluster.
> Now, when Node 1 re-joins the cluster, it will attempt to disable the 
> processor but won't be able to because the processor is still stopping. The 
> following will be in the logs:
> {code:java}
> 2018-05-16 15:21:50,986 WARN [Reconnect to Cluster] 
> org.apache.nifi.controller.ProcessorNode Processor cannot be disabled because 
> its state is set to STOPPING{code}
> So we now have a node in an inconsistent state.
> Additionally, if we now go to Node 1 in our browser and unselect all 
> components, and attempt to STOP the process group, the request that is 
> replicated attempts to stop the DebugFlow processor. Node 2 will now fail to 
> stop the processor because the processor is disabled. As a result, Node 2 
> will now be kicked out of the cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5204) When node joins cluster, if a processor is stopping but cluster says the state is disabled, node ends up in inconsistent state

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479446#comment-16479446
 ] 

ASF GitHub Bot commented on NIFI-5204:
--

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/2713
  
Thanks @markap14! This has been merged to master.


> When node joins cluster, if a processor is stopping but cluster says the 
> state is disabled, node ends up in inconsistent state
> --
>
> Key: NIFI-5204
> URL: https://issues.apache.org/jira/browse/NIFI-5204
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Critical
>
> In order to make this "easy" to replicate, I did the following:
> 1) Create a 2-node cluster.
>  2) On both nodes, update nifi.properties to set 
> "nifi.variable.registry.properties" to "1.properties"
>  3) On both nodes, create 1.properties in $NIFI_HOME. For first node, set 
> "sleep=2 mins" and for second node, set "sleep=0 millis"
>  4) Update DebugFlow to support expression language for the "@OnStopped Pause 
> Time"
>  5) Configure flow with a DebugFlow processor. Can auto-terminate 
> relationships and set run period to "10 secs."Set "@OnStopped Pause time" to 
> "${sleep}"
>  6) Disable DebugFlow processor.
>  7) Disconnected Node 1.
>  8) Go to Node 1 in browser and Start DebugFlow.
>  9) Stop DebugFlow.
>  10) While processor is still "stopping", go back Node 2 in browser and 
> request that Node 1 re-join the cluster.
> Now, when Node 1 re-joins the cluster, it will attempt to disable the 
> processor but won't be able to because the processor is still stopping. The 
> following will be in the logs:
> {code:java}
> 2018-05-16 15:21:50,986 WARN [Reconnect to Cluster] 
> org.apache.nifi.controller.ProcessorNode Processor cannot be disabled because 
> its state is set to STOPPING{code}
> So we now have a node in an inconsistent state.
> Additionally, if we now go to Node 1 in our browser and unselect all 
> components, and attempt to STOP the process group, the request that is 
> replicated attempts to stop the DebugFlow processor. Node 2 will now fail to 
> stop the processor because the processor is disabled. As a result, Node 2 
> will now be kicked out of the cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5204) When node joins cluster, if a processor is stopping but cluster says the state is disabled, node ends up in inconsistent state

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479442#comment-16479442
 ] 

ASF subversion and git services commented on NIFI-5204:
---

Commit 2afbf963812bccdec8c997e6c63a93c644016769 in nifi's branch 
refs/heads/master from [~markap14]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2afbf96 ]

NIFI-5204: Ensure that verifyCanStop throws ISE if component is disabled
NIFI-5204: If processor joins cluster and inherits 'disabled' state but is 
still stopping, ensure that the state becomes disabled when the processor 
finishes stopping
This closes #2713


> When node joins cluster, if a processor is stopping but cluster says the 
> state is disabled, node ends up in inconsistent state
> --
>
> Key: NIFI-5204
> URL: https://issues.apache.org/jira/browse/NIFI-5204
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Critical
>
> In order to make this "easy" to replicate, I did the following:
> 1) Create a 2-node cluster.
>  2) On both nodes, update nifi.properties to set 
> "nifi.variable.registry.properties" to "1.properties"
>  3) On both nodes, create 1.properties in $NIFI_HOME. For first node, set 
> "sleep=2 mins" and for second node, set "sleep=0 millis"
>  4) Update DebugFlow to support expression language for the "@OnStopped Pause 
> Time"
>  5) Configure flow with a DebugFlow processor. Can auto-terminate 
> relationships and set run period to "10 secs."Set "@OnStopped Pause time" to 
> "${sleep}"
>  6) Disable DebugFlow processor.
>  7) Disconnected Node 1.
>  8) Go to Node 1 in browser and Start DebugFlow.
>  9) Stop DebugFlow.
>  10) While processor is still "stopping", go back Node 2 in browser and 
> request that Node 1 re-join the cluster.
> Now, when Node 1 re-joins the cluster, it will attempt to disable the 
> processor but won't be able to because the processor is still stopping. The 
> following will be in the logs:
> {code:java}
> 2018-05-16 15:21:50,986 WARN [Reconnect to Cluster] 
> org.apache.nifi.controller.ProcessorNode Processor cannot be disabled because 
> its state is set to STOPPING{code}
> So we now have a node in an inconsistent state.
> Additionally, if we now go to Node 1 in our browser and unselect all 
> components, and attempt to STOP the process group, the request that is 
> replicated attempts to stop the DebugFlow processor. Node 2 will now fail to 
> stop the processor because the processor is disabled. As a result, Node 2 
> will now be kicked out of the cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5204) When node joins cluster, if a processor is stopping but cluster says the state is disabled, node ends up in inconsistent state

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479251#comment-16479251
 ] 

ASF GitHub Bot commented on NIFI-5204:
--

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/2713
  
Will review...


> When node joins cluster, if a processor is stopping but cluster says the 
> state is disabled, node ends up in inconsistent state
> --
>
> Key: NIFI-5204
> URL: https://issues.apache.org/jira/browse/NIFI-5204
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Critical
>
> In order to make this "easy" to replicate, I did the following:
> 1) Create a 2-node cluster.
>  2) On both nodes, update nifi.properties to set 
> "nifi.variable.registry.properties" to "1.properties"
>  3) On both nodes, create 1.properties in $NIFI_HOME. For first node, set 
> "sleep=2 mins" and for second node, set "sleep=0 millis"
>  4) Update DebugFlow to support expression language for the "@OnStopped Pause 
> Time"
>  5) Configure flow with a DebugFlow processor. Can auto-terminate 
> relationships and set run period to "10 secs."Set "@OnStopped Pause time" to 
> "${sleep}"
>  6) Disable DebugFlow processor.
>  7) Disconnected Node 1.
>  8) Go to Node 1 in browser and Start DebugFlow.
>  9) Stop DebugFlow.
>  10) While processor is still "stopping", go back Node 2 in browser and 
> request that Node 1 re-join the cluster.
> Now, when Node 1 re-joins the cluster, it will attempt to disable the 
> processor but won't be able to because the processor is still stopping. The 
> following will be in the logs:
> {code:java}
> 2018-05-16 15:21:50,986 WARN [Reconnect to Cluster] 
> org.apache.nifi.controller.ProcessorNode Processor cannot be disabled because 
> its state is set to STOPPING{code}
> So we now have a node in an inconsistent state.
> Additionally, if we now go to Node 1 in our browser and unselect all 
> components, and attempt to STOP the process group, the request that is 
> replicated attempts to stop the DebugFlow processor. Node 2 will now fail to 
> stop the processor because the processor is disabled. As a result, Node 2 
> will now be kicked out of the cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5204) When node joins cluster, if a processor is stopping but cluster says the state is disabled, node ends up in inconsistent state

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478209#comment-16478209
 ] 

ASF GitHub Bot commented on NIFI-5204:
--

GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/2713

NIFI-5204: If processor joins cluster and inherits 'disabled' state but is 
still stopping, ensure that the state becomes disabled when the processor 
finishes stopping and that the state cannot change in the meantime

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-5204

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2713.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2713


commit 34dcc69799072b3852562a61deea75dd9d2efb2f
Author: Mark Payne 
Date:   2018-05-16T20:06:00Z

NIFI-5204: Ensure that verifyCanStop throws ISE if component is disabled

commit a788bb7c30748562ae969ff36fae6b336a316011
Author: Mark Payne 
Date:   2018-05-16T21:56:20Z

NIFI-5204: If processor joins cluster and inherits 'disabled' state but is 
still stopping, ensure that the state becomes disabled when the processor 
finishes stopping




> When node joins cluster, if a processor is stopping but cluster says the 
> state is disabled, node ends up in inconsistent state
> --
>
> Key: NIFI-5204
> URL: https://issues.apache.org/jira/browse/NIFI-5204
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Critical
>
> In order to make this "easy" to replicate, I did the following:
> 1) Create a 2-node cluster.
>  2) On both nodes, update nifi.properties to set 
> "nifi.variable.registry.properties" to "1.properties"
>  3) On both nodes, create 1.properties in $NIFI_HOME. For first node, set 
> "sleep=2 mins" and for second node, set "sleep=0 millis"
>  4) Update DebugFlow to support expression language for the "@OnStopped Pause 
> Time"
>  5) Configure flow with a DebugFlow processor. Can auto-terminate 
> relationships and set run period to "10 secs."Set "@OnStopped Pause time" to 
> "${sleep}"
>  6) Disable DebugFlow processor.
>  7) Disconnected Node 1.
>  8) Go to Node 1 in browser and Start DebugFlow.
>  9) Stop DebugFlow.
>  10) While processor is still "stopping", go back Node 2 in browser and 
> request that Node 1 re-join the cluster.
> Now, when Node 1 re-joins the cluster, it will attempt to disable the 
> processor but won't be able to because the processor is still stopping. The 
> following will be in the logs:
> {code:java}
> 2018-05-16 15:21:50,986 WARN [Reconnect to Cluster] 
> org.apache.nifi.controller.ProcessorNode Processor cannot be disabled because 
> its state is set to STOPPING{code}
> So we now have a node in