Hello,
The reason for the error about the properties being sensitive is due
to the ghost controller services... since a ghost controller service
means we are missing the real definition of the component, NiFi
doesn't know if the properties are actually sensitive or not, so it
makes all properties on the ghost component sensitive to be cautious.
If you just upgraded from one version of nifi to the next, then you
shouldn't have ghost components. Ghost components happen in one of two
cases...
a) your flow.xml.gz references a component that doesn't exist in any of the NARs
b) your flow.xml.gz references a version of a component say X, and
there are multiple versions available in the NARs, but none of them
are X, for example versions Y and Z are available, but not X, so it
doesn't know which one is correct.
Thanks,
Bryan
On Sun, Sep 19, 2021 at 3:05 PM <christian.gump...@ezv.admin.ch> wrote:
>
> Dear Nifi experts,
>
>
>
> we are running a single-node Nifi installation with Nifi version 1.11.2 and
> recently started to look into the upgrade to Nifi 1.14.0. While doing so we
> observe one very odd behaviour when starting the new Nifi version with our
> existing flow:
>
>
>
> During startup the Nifi initialization fails as some properties from
> controller services or processors are set using parameters. According to the
> error message (stack trace below) the property is considered being sensitive
> (even though the documentation states it is not, e.g. “Database Connection
> URL” in DBCPConnectionPool ) while the parameter is not sensitive. In our
> humble opinion the error lies in the wrong assumption that the property is
> sensitive.
>
>
>
> One thing we noticed is that there multiple instances of
> GhostControllerServices being created because the old NAR versions are not
> found anymore. We assume that this is normal behaviour.
>
>
>
> Things we have tried:
>
> · Starting Nifi with a clean/empty flow -> works
>
> · “Hack” the flow XML so that the parameter is defined as sensitive
> -> kind of works as it falls over at the next parameter (we didn’t yet test
> to make them all sensitive)
>
>
>
> Any help or suggestion is greatly appreciated,
>
> Christian
>
>
>
> Stacktrace:
>
>
>
> 2021-09-17 15:54:52,575 DEBUG [main] o.a.n.c.v.StandardValidationTrigger
> Triggered to perform validation on
> StandardControllerServiceNode[service=GhostControllerService[id=c86a30d7-ddbe-1257-57a4-32cd7854dd56,
> type=org.apache.nifi.dbcp.DBCPConnectionPool], name=TeradataConnection,
> active=false] asynchronously but flow is not yet initialized so will ignore
> validation
>
> 2021-09-17 15:54:52,575 WARN [main] org.eclipse.jetty.webapp.WebAppContext
> Failed startup of context
> o.e.j.w.WebAppContext@1ab268bd{nifi-api,/nifi-api,file:///appl/nifi/nifi-1.14.0/work/jetty/nifi-web-api-1.14.0.war/webapp/,UNAVAILABLE}{/appl/nifi/nifi-current/work/nar/extensions/nifi-server-nar-1.14.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.14.0.war}
>
> org.apache.nifi.controller.serialization.FlowSynchronizationException:
> java.lang.IllegalArgumentException: The property 'Database Connection URL' is
> a sensitive property, so it can only reference Parameters that are sensitive.
>
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:306)
>
> at
> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1469)
>
> at
> org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89)
>
> at
> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810)
>
> at
> org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:539)
>
> at
> org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:67)
>
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068)
>
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
>
> at
> org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997)
>
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746)
>
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
>
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
>
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
>
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911)
>
> at
> org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
>
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
>
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
>
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
>
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
>
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
>
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
>
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
>
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
>
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
>
> at
> org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426)
>
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
>
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
>
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
>
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
>
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
>
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
>
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
>
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
>
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
>
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
>
> at org.eclipse.jetty.server.Server.start(Server.java:423)
>
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
>
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
>
> at org.eclipse.jetty.server.Server.doStart(Server.java:387)
>
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
>
> at
> org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1129)
>
> at org.apache.nifi.NiFi.<init>(NiFi.java:159)
>
> at org.apache.nifi.NiFi.<init>(NiFi.java:71)
>
> at org.apache.nifi.NiFi.main(NiFi.java:303)
>
> Caused by: java.lang.IllegalArgumentException: The property 'Database
> Connection URL' is a sensitive property, so it can only reference Parameters
> that are sensitive.
>
> at
> org.apache.nifi.controller.AbstractComponentNode.verifyCanUpdateProperties(AbstractComponentNode.java:301)
>
> at
> org.apache.nifi.controller.AbstractComponentNode.setProperties(AbstractComponentNode.java:196)
>
> at
> org.apache.nifi.controller.ComponentNode.setProperties(ComponentNode.java:61)
>
> at
> org.apache.nifi.controller.service.ControllerServiceLoader.configureControllerService(ControllerServiceLoader.java:219)
>
> at
> org.apache.nifi.controller.service.ControllerServiceLoader.loadControllerServices(ControllerServiceLoader.java:123)
>
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.addControllerServices(StandardFlowSynchronizer.java:1391)
>
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1334)
>
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.addNestedProcessGroups(StandardFlowSynchronizer.java:1351)
>
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1340)
>
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.updateFlow(StandardFlowSynchronizer.java:432)
>
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:283)
>
> ... 43 common frames omitted
>
>
>
> Christian Gumpert
>
> Leiter BI & Analytics
>
> Lead Data Scientist
>
> Product Owner Team Lagrev
>
>
>
> Eidgenössisches Finanzdepartement EFD
>
> Eidgenössische Zollverwaltung EZV
>
> Direktionsbereich Risikoanalyse und Analytik
>
>
>
> Taubenstrasse 16, 3003 Bern
>
> christian.gump...@ezv.admin.ch
>
> www.ezv.admin.ch
>
>
