[jira] [Commented] (NIFI-11189) Restarting NiFi after failing to upgrade flow can cause NiFi to fail startup
[
https://issues.apache.org/jira/browse/NIFI-11189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697648#comment-17697648
]
Joe Witt commented on NIFI-11189:
-
[~markap14][~bsimon]once a user hits this what do they do to get back on track?
> Restarting NiFi after failing to upgrade flow can cause NiFi to fail startup
>
>
> Key: NIFI-11189
> URL: https://issues.apache.org/jira/browse/NIFI-11189
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 2.0.0, 1.21.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When a user updates a group that is under Version Control from one version to
> another, sometimes the destination of a connection now needs to point to a
> new component. And the current destination gets deleted, while the new
> component hasn't yet been created. To handle this, we create a temporary
> funnel and set the connection’s destination to that funnel.
>
> Once all components are created, we then move the connections to their
> intended destination. Then delete the temporary funnel.
>
> We have an issue, however. If, for some reason, we fail to complete the flow
> upgrade, that funnel may remain. We are not handling this case where this
> temporary funnel already exists in the flow definition very well on restart.
> This can result in an error such as:
> {code:java}
> 2023-02-15 16:40:03,347 WARN [main] org.eclipse.jetty.webapp.WebAppContext
> Failed startup of context
> o.e.j.w.WebAppContext@345af277{nifi-api,/nifi-api,file:///opt/nifi-1.18.0.2.1.5.1001-1/work/jetty/nifi-web-api-1.18.0.2.1.5.1001-1.war/webapp/,UNAVAILABLE}{./work/nar/extensions/nifi-server-nar-1.18.0.2.1.5.1001-1.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.18.0.2.1.5.1001-1.war}
> org.apache.nifi.controller.serialization.FlowSynchronizationException:
> java.lang.IllegalArgumentException: Connection has a destination with
> identifier c594bee4-b49e-34a3-8795-732d890df61f but no component could be
> found in the Process Group with a corresponding identifier
> at
> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:454)
> at
> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.sync(VersionedFlowSynchronizer.java:205)
> at
> org.apache.nifi.controller.serialization.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:42)
> at
> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1525)
> at
> org.apache.nifi.persistence.StandardFlowConfigurationDAO.load(StandardFlowConfigurationDAO.java:104)
> at
> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:837)
> at
> org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:558)
> at
> org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:67)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
> at
> org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002)
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:765)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
> at
>
[jira] [Commented] (NIFI-11189) Restarting NiFi after failing to upgrade flow can cause NiFi to fail startup
[
https://issues.apache.org/jira/browse/NIFI-11189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697649#comment-17697649
]
Joe Witt commented on NIFI-11189:
-
https://apachenifi.slack.com/archives/C0L9VCD47/p1678230074353839
> Restarting NiFi after failing to upgrade flow can cause NiFi to fail startup
>
>
> Key: NIFI-11189
> URL: https://issues.apache.org/jira/browse/NIFI-11189
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 2.0.0, 1.21.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When a user updates a group that is under Version Control from one version to
> another, sometimes the destination of a connection now needs to point to a
> new component. And the current destination gets deleted, while the new
> component hasn't yet been created. To handle this, we create a temporary
> funnel and set the connection’s destination to that funnel.
>
> Once all components are created, we then move the connections to their
> intended destination. Then delete the temporary funnel.
>
> We have an issue, however. If, for some reason, we fail to complete the flow
> upgrade, that funnel may remain. We are not handling this case where this
> temporary funnel already exists in the flow definition very well on restart.
> This can result in an error such as:
> {code:java}
> 2023-02-15 16:40:03,347 WARN [main] org.eclipse.jetty.webapp.WebAppContext
> Failed startup of context
> o.e.j.w.WebAppContext@345af277{nifi-api,/nifi-api,file:///opt/nifi-1.18.0.2.1.5.1001-1/work/jetty/nifi-web-api-1.18.0.2.1.5.1001-1.war/webapp/,UNAVAILABLE}{./work/nar/extensions/nifi-server-nar-1.18.0.2.1.5.1001-1.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.18.0.2.1.5.1001-1.war}
> org.apache.nifi.controller.serialization.FlowSynchronizationException:
> java.lang.IllegalArgumentException: Connection has a destination with
> identifier c594bee4-b49e-34a3-8795-732d890df61f but no component could be
> found in the Process Group with a corresponding identifier
> at
> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:454)
> at
> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.sync(VersionedFlowSynchronizer.java:205)
> at
> org.apache.nifi.controller.serialization.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:42)
> at
> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1525)
> at
> org.apache.nifi.persistence.StandardFlowConfigurationDAO.load(StandardFlowConfigurationDAO.java:104)
> at
> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:837)
> at
> org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:558)
> at
> org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:67)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
> at
> org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002)
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:765)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
> at
>
[jira] [Commented] (NIFI-11189) Restarting NiFi after failing to upgrade flow can cause NiFi to fail startup
[
https://issues.apache.org/jira/browse/NIFI-11189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689791#comment-17689791
]
ASF subversion and git services commented on NIFI-11189:
Commit 6f2b9e5f65198b0eb3870d9ae268429803f31a3f in nifi's branch
refs/heads/support/nifi-1.x from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6f2b9e5f65 ]
NIFI-11189: When synchronizing a ProcessGroup to match a VersionedProcessGroup,
do not remove the temporary funnel until the very end. This is important if the
temporary funnel already exists in the flow on startup
Signed-off-by: Bence Simon
This closes #6963
> Restarting NiFi after failing to upgrade flow can cause NiFi to fail startup
>
>
> Key: NIFI-11189
> URL: https://issues.apache.org/jira/browse/NIFI-11189
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 2.0.0, 1.21.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When a user updates a group that is under Version Control from one version to
> another, sometimes the destination of a connection now needs to point to a
> new component. And the current destination gets deleted, while the new
> component hasn't yet been created. To handle this, we create a temporary
> funnel and set the connection’s destination to that funnel.
>
> Once all components are created, we then move the connections to their
> intended destination. Then delete the temporary funnel.
>
> We have an issue, however. If, for some reason, we fail to complete the flow
> upgrade, that funnel may remain. We are not handling this case where this
> temporary funnel already exists in the flow definition very well on restart.
> This can result in an error such as:
> {code:java}
> 2023-02-15 16:40:03,347 WARN [main] org.eclipse.jetty.webapp.WebAppContext
> Failed startup of context
> o.e.j.w.WebAppContext@345af277{nifi-api,/nifi-api,file:///opt/nifi-1.18.0.2.1.5.1001-1/work/jetty/nifi-web-api-1.18.0.2.1.5.1001-1.war/webapp/,UNAVAILABLE}{./work/nar/extensions/nifi-server-nar-1.18.0.2.1.5.1001-1.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.18.0.2.1.5.1001-1.war}
> org.apache.nifi.controller.serialization.FlowSynchronizationException:
> java.lang.IllegalArgumentException: Connection has a destination with
> identifier c594bee4-b49e-34a3-8795-732d890df61f but no component could be
> found in the Process Group with a corresponding identifier
> at
> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:454)
> at
> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.sync(VersionedFlowSynchronizer.java:205)
> at
> org.apache.nifi.controller.serialization.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:42)
> at
> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1525)
> at
> org.apache.nifi.persistence.StandardFlowConfigurationDAO.load(StandardFlowConfigurationDAO.java:104)
> at
> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:837)
> at
> org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:558)
> at
> org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:67)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
> at
> org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002)
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:765)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
> at
>
[jira] [Commented] (NIFI-11189) Restarting NiFi after failing to upgrade flow can cause NiFi to fail startup
[
https://issues.apache.org/jira/browse/NIFI-11189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689761#comment-17689761
]
ASF subversion and git services commented on NIFI-11189:
Commit fed175d46f794cffae8695df6eb63adab4e8a27a in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=fed175d46f ]
NIFI-11189: When synchronizing a ProcessGroup to match a VersionedProcessGroup,
do not remove the temporary funnel until the very end. This is important if the
temporary funnel already exists in the flow on startup
Signed-off-by: Bence Simon
This closes #6963
> Restarting NiFi after failing to upgrade flow can cause NiFi to fail startup
>
>
> Key: NIFI-11189
> URL: https://issues.apache.org/jira/browse/NIFI-11189
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 2.0.0, 1.21.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When a user updates a group that is under Version Control from one version to
> another, sometimes the destination of a connection now needs to point to a
> new component. And the current destination gets deleted, while the new
> component hasn't yet been created. To handle this, we create a temporary
> funnel and set the connection’s destination to that funnel.
>
> Once all components are created, we then move the connections to their
> intended destination. Then delete the temporary funnel.
>
> We have an issue, however. If, for some reason, we fail to complete the flow
> upgrade, that funnel may remain. We are not handling this case where this
> temporary funnel already exists in the flow definition very well on restart.
> This can result in an error such as:
> {code:java}
> 2023-02-15 16:40:03,347 WARN [main] org.eclipse.jetty.webapp.WebAppContext
> Failed startup of context
> o.e.j.w.WebAppContext@345af277{nifi-api,/nifi-api,file:///opt/nifi-1.18.0.2.1.5.1001-1/work/jetty/nifi-web-api-1.18.0.2.1.5.1001-1.war/webapp/,UNAVAILABLE}{./work/nar/extensions/nifi-server-nar-1.18.0.2.1.5.1001-1.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.18.0.2.1.5.1001-1.war}
> org.apache.nifi.controller.serialization.FlowSynchronizationException:
> java.lang.IllegalArgumentException: Connection has a destination with
> identifier c594bee4-b49e-34a3-8795-732d890df61f but no component could be
> found in the Process Group with a corresponding identifier
> at
> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:454)
> at
> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.sync(VersionedFlowSynchronizer.java:205)
> at
> org.apache.nifi.controller.serialization.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:42)
> at
> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1525)
> at
> org.apache.nifi.persistence.StandardFlowConfigurationDAO.load(StandardFlowConfigurationDAO.java:104)
> at
> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:837)
> at
> org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:558)
> at
> org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:67)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
> at
> org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002)
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:765)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
> at
>
