Re: Custom NAR interfering with BundleUtils.findBundleForType

[Scott Wagner]

To provide closure to this thread:

Bryan, the exception indeed was being thrown there, but the root cause 
of that was because "Ghost" processors were being created in the place 
of the actual processors because my custom NAR was confusing 
BuildUtils.findBundleForType.


Joe, thank you very much for the suggestion of what to look into.  We 
had copied most of the InvokeHTTP processor into our NAR with some 
modifications, but it was still including references to classes in 
nifi-standard-processors which was being dragged in through the pom.  I 
was able to extricate the 2 classes I need and copy them into my own 
project to at least get NiFi up and running; while this is not the 
correct permanent solution, I was able to move past the startup problems.


Thanks for being part of such a great and responsive community.

- Scott


Bryan Bende 
Wednesday, July 5, 2017 12:07 PM
Scott,

Thanks for providing the stacktrace... do any of your custom
processors use the @DefaultSchedule annotation? and if so, do any of
them set the tasks to a number less than 1?

The exception you are getting is from some code that is preventing
using 0 or negative number of tasks for a processor that is not
scheduled for event driven, basically meaning event driven is the only
one where it would make sense to have less than 1 task.

What Joe mentioned about including other standard processors in your
NAR could very well still be a problem, but that stacktrace might be
something else.

-Bryan

Scott Wagner 
Wednesday, July 5, 2017 11:44 AM
Hi Joe,

We are extending AbstractProcessor for our processors, and 
AbstractControllerService for our controller service.  However, we did 
include the InvokeHTTP processor with some modifications that are 
referencing some other classes that are in the 
nifi-processors-standard JAR.  I will look into breaking those out to 
remove that dependency.


The actual error that we are getting is below:

2017-07-04 12:28:29,076 WARN [main] 
org.apache.nifi.web.server.JettyServer Failed to start web server... 
shutting down.
org.apache.nifi.controller.serialization.FlowSynchronizationException: 
java.lang.IllegalArgumentException
at 
org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:426)
at 
org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1576)
at 
org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:84)
at 
org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:722)
at 
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:533)
at 
org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72)
at 
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:876)
at 
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:532)
at 
org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:839)
at 
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:344)
at 
org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1480)
at 
org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1442)
at 
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:799)
at 
org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:261)
at 
org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:540)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
at 
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
at 
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:290)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)

at org.eclipse.jetty.server.Server.start(Server.java:452)
at 

Re: Custom NAR interfering with BundleUtils.findBundleForType

[Bryan Bende]

Scott,

Thanks for providing the stacktrace... do any of your custom
processors use the @DefaultSchedule annotation? and if so, do any of
them set the tasks to a number less than 1?

The exception you are getting is from some code that is preventing
using 0 or negative number of tasks for a processor that is not
scheduled for event driven, basically meaning event driven is the only
one where it would make sense to have less than 1 task.

What Joe mentioned about including other standard processors in your
NAR could very well still be a problem, but that stacktrace might be
something else.

-Bryan


On Wed, Jul 5, 2017 at 12:44 PM, Scott Wagner  wrote:
> Hi Joe,
>
> We are extending AbstractProcessor for our processors, and
> AbstractControllerService for our controller service.  However, we did
> include the InvokeHTTP processor with some modifications that are
> referencing some other classes that are in the nifi-processors-standard JAR.
> I will look into breaking those out to remove that dependency.
>
> The actual error that we are getting is below:
>
> 2017-07-04 12:28:29,076 WARN [main] org.apache.nifi.web.server.JettyServer
> Failed to start web server... shutting down.
> org.apache.nifi.controller.serialization.FlowSynchronizationException:
> java.lang.IllegalArgumentException
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:426)
> at
> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1576)
> at
> org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:84)
> at
> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:722)
> at
> org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:533)
> at
> org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:876)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:532)
> at
> org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:839)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:344)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1480)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1442)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:799)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:261)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:540)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
> at
> org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:290)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
> at org.eclipse.jetty.server.Server.start(Server.java:452)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
> at org.eclipse.jetty.server.Server.doStart(Server.java:419)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.apache.nifi.web.server.JettyServer.start(JettyServer.java:705)
> at org.apache.nifi.NiFi.(NiFi.java:160)
> at org.apache.nifi.NiFi.main(NiFi.java:267)
> Caused by: java.lang.IllegalArgumentException: null
> at
> org.apache.nifi.controller.StandardProcessorNode.setMaxConcurrentTasks(StandardProcessorNode.java:620)
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.updateProcessor(StandardFlowSynchronizer.java:985)
> at
> 

Re: Custom NAR interfering with BundleUtils.findBundleForType

[Scott Wagner]

Hi Joe,

We are extending AbstractProcessor for our processors, and 
AbstractControllerService for our controller service.  However, we did 
include the InvokeHTTP processor with some modifications that are 
referencing some other classes that are in the nifi-processors-standard 
JAR.  I will look into breaking those out to remove that dependency.


The actual error that we are getting is below:

2017-07-04 12:28:29,076 WARN [main] 
org.apache.nifi.web.server.JettyServer Failed to start web server... 
shutting down.
org.apache.nifi.controller.serialization.FlowSynchronizationException: 
java.lang.IllegalArgumentException
at 
org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:426)
at 
org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1576)
at 
org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:84)
at 
org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:722)
at 
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:533)
at 
org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72)
at 
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:876)
at 
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:532)
at 
org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:839)
at 
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:344)
at 
org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1480)
at 
org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1442)
at 
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:799)
at 
org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:261)
at 
org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:540)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
at 
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
at 
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:290)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)

at org.eclipse.jetty.server.Server.start(Server.java:452)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
at 
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)

at org.eclipse.jetty.server.Server.doStart(Server.java:419)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at 
org.apache.nifi.web.server.JettyServer.start(JettyServer.java:705)

at org.apache.nifi.NiFi.(NiFi.java:160)
at org.apache.nifi.NiFi.main(NiFi.java:267)
Caused by: java.lang.IllegalArgumentException: null
at 
org.apache.nifi.controller.StandardProcessorNode.setMaxConcurrentTasks(StandardProcessorNode.java:620)
at 
org.apache.nifi.controller.StandardFlowSynchronizer.updateProcessor(StandardFlowSynchronizer.java:985)
at 
org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1064)
at 
org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1183)
at 
org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1183)
at 
org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1183)
at 
org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:312)

... 33 common frames omitted



Joe Witt 
Wednesday, July 5, 2017 10:57 AM
Scott

In your custom NAR are you extending some other processor/component
such as one found in standard processors? You'll really need to break
that chain and not pull in the same 

Re: Custom NAR interfering with BundleUtils.findBundleForType

[Joe Witt]

Scott

In your custom NAR are you extending some other processor/component
such as one found in standard processors?  You'll really need to break
that chain and not pull in the same components.

Can you share the actual error being provided at startup/

Thanks
joe

On Wed, Jul 5, 2017 at 11:53 AM, Scott Wagner  wrote:
> Hey all,
>
> I have been running on NiFi 1.1.1 for a while, and just started working
> on migrating to 1.3.0.  However, when I tried starting up, I was getting
> unhandled exceptions during startup of Jetty.  I have traced this down to
> something being wrong with a custom NAR file that we built for 1.1.1.
>
> The problem is when BundleUtils.findBundleForType is being called, it is
> finding 2 bundles for every internal NiFi processor class - the correct NAR
> that it comes with in 1.3.0, and my custom NAR file as well.  I'm sure that
> I have done something wrong in the packaging for my custom NAR file, but I
> don't know what it could be.  It was originally built from the NiFi NAR
> archetype late last year.
>
> If anyone can point in me in the direction of what I'm doing wrong, I
> would greatly appreciate it.  I'm not a maven/packaging expert by any means
> so no advice is too basic.
>
> Thanks!
>
> - Scott