Mark Payne created NIFI-11192:
---------------------------------
Summary: If Port moved from parent to child group or vice versa
between flow versions, version change can leave nifi in bad state
Key: NIFI-11192
URL: https://issues.apache.org/jira/browse/NIFI-11192
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Mark Payne
Fix For: 2.0.0, 1.21.0
To reproduce:
Create a Process Group, Parent.
Inside of Parent, create:
* A processor, for example, ReplaceText
* A Process Group, Child.
Inside of Child, create:
* An Input Port, In
* A Processor, for example, UpdateAttribute
* A connection between the two components
Then connect ReplaceText to the Input Port, In.
Save the Parent PG flow as Version 1 of a flow.
Now, create a new Processor, say RouteOnAttribute, within the Parent PG.
Move the destination of the connection from the Input Port to RouteOnAttribute.
Step into the Child PG. Select all components, right-click, and choose "Move to
Parent"
Save Parent PG as Version 2 of the flow.
Now, attempt to Change Version on the Parent Group. Change the version to
Version 1.
The version change will fail with an error: "Failed to perform update flow
request due to 42fb2904-c774-359b-5368-2e48b60ac02d is the destination of
another component" and the logs will have a stack trace:
{code:java}
2023-02-16 15:18:28,830 ERROR [Process Group Update Thread-1]
o.apache.nifi.web.api.FlowUpdateResource Failed to perform update flow request
java.lang.IllegalStateException: 42fb2904-c774-359b-5368-2e48b60ac02d is the
destination of another component
at
org.apache.nifi.controller.AbstractPort.verifyCanDelete(AbstractPort.java:562)
at
org.apache.nifi.controller.AbstractPort.verifyCanDelete(AbstractPort.java:542)
at
org.apache.nifi.groups.StandardProcessGroup.removeInputPort(StandardProcessGroup.java:637)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.removeMissingComponents(StandardVersionedComponentSynchronizer.java:948)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.removeMissingInputPorts(StandardVersionedComponentSynchronizer.java:873)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:410)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:260)
at
org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:556)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:255)
at
org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3972)
at
org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3952)
at
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:435)
at
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
at
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:308)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
at
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
at
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$2edbbc7e.updateProcessGroupFlow(<generated>)
at
org.apache.nifi.web.StandardNiFiServiceFacade$13.update(StandardNiFiServiceFacade.java:5848)
at
org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:130)
at
org.apache.nifi.web.StandardNiFiServiceFacade.updateProcessGroupContents(StandardNiFiServiceFacade.java:5843)
at
org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
at
org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithWriteLock(NiFiServiceFacadeLock.java:179)
at
org.apache.nifi.web.NiFiServiceFacadeLock.updateLock(NiFiServiceFacadeLock.java:66)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
at
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
at
org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$cd414c69.updateProcessGroupContents(<generated>)
at
org.apache.nifi.web.api.VersionsResource.performUpdateFlow(VersionsResource.java:1221)
at
org.apache.nifi.web.api.VersionsResource.performUpdateFlow(VersionsResource.java:85)
at
org.apache.nifi.web.api.FlowUpdateResource.updateFlow(FlowUpdateResource.java:428)
at
org.apache.nifi.web.api.FlowUpdateResource.lambda$submitFlowUpdateRequest$5(FlowUpdateResource.java:281)
at
org.apache.nifi.web.api.concurrent.AsyncRequestManager$2.run(AsyncRequestManager.java:117)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829) {code}
This now leaves NiFi in a bad state, where the flow is partially updated but
not fully. What's worse, though, is that it leaves the ReplaceText Processor
connected to an Input Port within the same Process Group, which is not legal.
At this point, the flow has not be written to the flow.json.gz/flow.xml.gz
because the update operation failed.
Now, move the ReplaceText Processor to the side a bit, in order to trigger the
flow to be written out again.
Now, attempt to restart NiFI, and it will fail to startup, yielding the
following error in the logs:
{code:java}
2023-02-16 15:23:10,770 WARN [main] org.eclipse.jetty.webapp.WebAppContext
Failed startup of context
o.e.j.w.WebAppContext@3fe8ad3f{nifi-api,/nifi-api,file:///Users/mpayne/devel/nifi/nifi-assembly/target/nifi-2.0.0-SNAPSHOT-bin/nifi-2.0.0-SNAPSHOT/work/jetty/nifi-web-api-2.0.0-SNAPSHOT.war/webapp/,UNAVAILABLE}{./work/nar/extensions/nifi-server-nar-2.0.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-2.0.0-SNAPSHOT.war}
org.apache.nifi.controller.serialization.FlowSynchronizationException:
java.lang.IllegalStateException: Cannot add Connection from
PROCESSOR[7a6913fa-3a97-3ce2-c128-d915258fae47] from Process Group
[5bd874fd-0186-1000-222a-d40ae014c2d3] to Process Group
[5bd874fd-0186-1000-222a-d40ae014c2d3] because its destination
[42fb2904-c774-359b-5368-2e48b60ac02d] is an Input Port but the Input Port does
not belong to a child Process Group
at
org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:448)
at
org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.sync(VersionedFlowSynchronizer.java:206)
at
org.apache.nifi.controller.serialization.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:42)
at
org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1638)
at
org.apache.nifi.persistence.StandardFlowConfigurationDAO.load(StandardFlowConfigurationDAO.java:104)
at
org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:817)
at
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:538)
at
org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:67)
at
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073)
at
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
at
org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002)
at
org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:765)
at
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
at
org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
at
org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
at
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
at
org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at
org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.server.Server.start(Server.java:423)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at org.eclipse.jetty.server.Server.doStart(Server.java:387)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:815)
at org.apache.nifi.NiFi.<init>(NiFi.java:172)
at org.apache.nifi.NiFi.<init>(NiFi.java:83)
at org.apache.nifi.NiFi.main(NiFi.java:332)
Caused by: java.lang.IllegalStateException: Cannot add Connection from
PROCESSOR[7a6913fa-3a97-3ce2-c128-d915258fae47] from Process Group
[5bd874fd-0186-1000-222a-d40ae014c2d3] to Process Group
[5bd874fd-0186-1000-222a-d40ae014c2d3] because its destination
[42fb2904-c774-359b-5368-2e48b60ac02d] is an Input Port but the Input Port does
not belong to a child Process Group
at
org.apache.nifi.groups.StandardProcessGroup.addConnection(StandardProcessGroup.java:1310)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.addConnection(StandardVersionedComponentSynchronizer.java:3312)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeConnections(StandardVersionedComponentSynchronizer.java:661)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:461)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.addProcessGroup(StandardVersionedComponentSynchronizer.java:1149)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:531)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:417)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:260)
at
org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:556)
at
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:255)
at
org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3972)
at
org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:439)
... 45 common frames omitted {code}
There are two issues here:
* We should not fail to update the version of the flow. We need to fix the
logic in order to properly handle the moving of a port between groups.
* If for some reason, we do fail to perform a Change Version request, we
currently leave the flow in a bad state. We should instead revert all changes
to the flow for which the change was attempted. This should prevent us from
getting the flow into a bad state, even if an update fails.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
