Hi Bryan, thanks for your feedback. Case b) is exactly what happened to us. After a clean install containing only the official NARs and leaving our custom patches aside, Nifi started without a problem.
Sorry for the noise, Christian Gumpert Leiter BI & Analytics Lead Data Scientist Product Owner Team Lagrev Eidgenössisches Finanzdepartement EFD Eidgenössische Zollverwaltung EZV Direktionsbereich Risikoanalyse und Analytik Taubenstrasse 16, 3003 Bern [email protected] www.ezv.admin.ch -----Ursprüngliche Nachricht----- Von: Bryan Bende <[email protected]> Gesendet: Montag, 20. September 2021 15:06 An: [email protected] Cc: Trappenberg Jonas BIT <[email protected]>; Schneider Basil EZV <[email protected]> Betreff: Re: Processor properties considered sensitive when migrating to Nifi 1.14.0 Hello, The reason for the error about the properties being sensitive is due to the ghost controller services... since a ghost controller service means we are missing the real definition of the component, NiFi doesn't know if the properties are actually sensitive or not, so it makes all properties on the ghost component sensitive to be cautious. If you just upgraded from one version of nifi to the next, then you shouldn't have ghost components. Ghost components happen in one of two cases... a) your flow.xml.gz references a component that doesn't exist in any of the NARs b) your flow.xml.gz references a version of a component say X, and there are multiple versions available in the NARs, but none of them are X, for example versions Y and Z are available, but not X, so it doesn't know which one is correct. Thanks, Bryan On Sun, Sep 19, 2021 at 3:05 PM <[email protected]> wrote: > > Dear Nifi experts, > > > > we are running a single-node Nifi installation with Nifi version 1.11.2 and > recently started to look into the upgrade to Nifi 1.14.0. While doing so we > observe one very odd behaviour when starting the new Nifi version with our > existing flow: > > > > During startup the Nifi initialization fails as some properties from > controller services or processors are set using parameters. According to the > error message (stack trace below) the property is considered being sensitive > (even though the documentation states it is not, e.g. “Database Connection > URL” in DBCPConnectionPool ) while the parameter is not sensitive. In our > humble opinion the error lies in the wrong assumption that the property is > sensitive. > > > > One thing we noticed is that there multiple instances of > GhostControllerServices being created because the old NAR versions are not > found anymore. We assume that this is normal behaviour. > > > > Things we have tried: > > · Starting Nifi with a clean/empty flow -> works > > · “Hack” the flow XML so that the parameter is defined as sensitive > -> kind of works as it falls over at the next parameter (we didn’t yet test > to make them all sensitive) > > > > Any help or suggestion is greatly appreciated, > > Christian > > > > Stacktrace: > > > > 2021-09-17 15:54:52,575 DEBUG [main] > o.a.n.c.v.StandardValidationTrigger Triggered to perform validation on > StandardControllerServiceNode[service=GhostControllerService[id=c86a30 > d7-ddbe-1257-57a4-32cd7854dd56, > type=org.apache.nifi.dbcp.DBCPConnectionPool], > name=TeradataConnection, active=false] asynchronously but flow is not > yet initialized so will ignore validation > > 2021-09-17 15:54:52,575 WARN [main] > org.eclipse.jetty.webapp.WebAppContext Failed startup of context > o.e.j.w.WebAppContext@1ab268bd{nifi-api,/nifi-api,file:///appl/nifi/ni > fi-1.14.0/work/jetty/nifi-web-api-1.14.0.war/webapp/,UNAVAILABLE}{/app > l/nifi/nifi-current/work/nar/extensions/nifi-server-nar-1.14.0.nar-unp > acked/NAR-INF/bundled-dependencies/nifi-web-api-1.14.0.war} > > org.apache.nifi.controller.serialization.FlowSynchronizationException: > java.lang.IllegalArgumentException: The property 'Database Connection URL' is > a sensitive property, so it can only reference Parameters that are sensitive. > > at > org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowS > ynchronizer.java:306) > > at > org.apache.nifi.controller.FlowController.synchronize(FlowController.j > ava:1469) > > at > org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(Stand > ardXMLFlowConfigurationDAO.java:89) > > at > org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardF > lowService.java:810) > > at > org.apache.nifi.controller.StandardFlowService.load(StandardFlowServic > e.java:539) > > at > org.apache.nifi.web.contextlistener.ApplicationStartupContextListener. > contextInitialized(ApplicationStartupContextListener.java:67) > > at > org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized > (ContextHandler.java:1068) > > at > org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized > (ServletContextHandler.java:572) > > at > org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(Con > textHandler.java:997) > > at > org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.jav > a:746) > > at > org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletCo > ntextHandler.java:379) > > at > org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java: > 1449) > > at > org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java > :1414) > > at > org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler > .java:911) > > at > org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContext > Handler.java:288) > > at > org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeC > ycle.java:73) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLif > eCycle.java:169) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerL > ifeCycle.java:117) > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandl > er.java:97) > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeC > ycle.java:73) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLif > eCycle.java:169) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerL > ifeCycle.java:110) > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandl > er.java:97) > > at > org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler. > java:426) > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeC > ycle.java:73) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLif > eCycle.java:169) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerL > ifeCycle.java:117) > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandl > er.java:97) > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeC > ycle.java:73) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLif > eCycle.java:169) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerL > ifeCycle.java:117) > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandl > er.java:97) > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeC > ycle.java:73) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLif > eCycle.java:169) > > at org.eclipse.jetty.server.Server.start(Server.java:423) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerL > ifeCycle.java:110) > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandl > er.java:97) > > at > org.eclipse.jetty.server.Server.doStart(Server.java:387) > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeC > ycle.java:73) > > at > org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1129) > > at org.apache.nifi.NiFi.<init>(NiFi.java:159) > > at org.apache.nifi.NiFi.<init>(NiFi.java:71) > > at org.apache.nifi.NiFi.main(NiFi.java:303) > > Caused by: java.lang.IllegalArgumentException: The property 'Database > Connection URL' is a sensitive property, so it can only reference Parameters > that are sensitive. > > at > org.apache.nifi.controller.AbstractComponentNode.verifyCanUpdateProper > ties(AbstractComponentNode.java:301) > > at > org.apache.nifi.controller.AbstractComponentNode.setProperties(Abstrac > tComponentNode.java:196) > > at > org.apache.nifi.controller.ComponentNode.setProperties(ComponentNode.j > ava:61) > > at > org.apache.nifi.controller.service.ControllerServiceLoader.configureCo > ntrollerService(ControllerServiceLoader.java:219) > > at > org.apache.nifi.controller.service.ControllerServiceLoader.loadControl > lerServices(ControllerServiceLoader.java:123) > > at > org.apache.nifi.controller.StandardFlowSynchronizer.addControllerServi > ces(StandardFlowSynchronizer.java:1391) > > at > org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(St > andardFlowSynchronizer.java:1334) > > at > org.apache.nifi.controller.StandardFlowSynchronizer.addNestedProcessGr > oups(StandardFlowSynchronizer.java:1351) > > at > org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(St > andardFlowSynchronizer.java:1340) > > at > org.apache.nifi.controller.StandardFlowSynchronizer.updateFlow(Standar > dFlowSynchronizer.java:432) > > at > org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowS > ynchronizer.java:283) > > ... 43 common frames omitted > > > > Christian Gumpert > > Leiter BI & Analytics > > Lead Data Scientist > > Product Owner Team Lagrev > > > > Eidgenössisches Finanzdepartement EFD > > Eidgenössische Zollverwaltung EZV > > Direktionsbereich Risikoanalyse und Analytik > > > > Taubenstrasse 16, 3003 Bern > > [email protected] > > www.ezv.admin.ch > >
