You have to add another instance of the processor which should
generate the same stracktrace you sent earlier, except this time there
should be a second part to it with "Caused by.... " and then more of
the stacktrace that wasn't there before.
On Wed, Aug 14, 2019 at 4:41 PM Bimal Mehta <bimal...@gmail.com> wrote:
>
> Hi Bryan,
>
> I did what you said.
> This is what I got
>
> 2019-08-14 20:16:18,948 DEBUG [Validate Components Thread-3]
> o.a.n.controller.AbstractComponentNode Computed validation errors with
> Validation Context StandardValidationContext[componentId=
> 6fbe2407-7799-3908-f4c4-bf2f8940bf1e , properties={PropertyDescriptor[Header
> Line Count]=1, PropertyDescriptor[Enable processing]=${searchTerm}}]; results
> = ['Missing Processor' validated against 'Any Property' is invalid because
> Processor is of type org.apache.nifi.init.InitiateScan, but this is not a
> valid Processor type]
> 2019-08-14 20:16:18,946 DEBUG [Timer-Driven Process Thread-6]
> o.a.n.c.r.m.SecondPrecisionEventContainer Updated bin 39. Did NOT replace.
>
> On Wed, Aug 14, 2019 at 1:42 PM Bryan Bende <bbe...@gmail.com> wrote:
>>
>> Can you edit logback.xml and add the following, the get the stacktrace again?
>>
>> <logger name="org.apache.nifi.controller" level="DEBUG" />
>>
>> This should include a root cause exception which we are missing right now.
>>
>> I think it takes about 20-30 seconds for logback to pick up the edits
>> to logback.xml.
>>
>> On Wed, Aug 14, 2019 at 12:53 PM Bimal Mehta <bimal...@gmail.com> wrote:
>> >
>> > For the custom processor we have, we are extending the
>> > AbstractNifiProcessor.java.
>> > The processor is used to scan metadata of an incoming flow file.
>> > The error we get in logs is as below.
>> > 2019-08-13 23:21:21,529 ERROR [main] o.a.nifi.controller.ExtensionBuilder
>> > Could not create Processor of type org.apache.nifi.init.InitiateScan for
>> > ID 6fbe2407-7799-3908-f4c4-bf2f8940bf1e; creating "Ghost" implementation
>> > org.apache.nifi.controller.exception.ProcessorInstantiationException:
>> > org.apache.nifi.init.InitiateScan
>> > at
>> > org.apache.nifi.controller.ExtensionBuilder.createLoggableProcessor(ExtensionBuilder.java:425)
>> > at
>> > org.apache.nifi.controller.ExtensionBuilder.buildProcessor(ExtensionBuilder.java:191)
>> > at
>> > org.apache.nifi.controller.flow.StandardFlowManager.createProcessor(StandardFlowManager.java:298)
>> > at
>> > org.apache.nifi.controller.flow.StandardFlowManager.createProcessor(StandardFlowManager.java:274)
>> > at
>> > org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1262)
>> > at
>> > org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1389)
>> > at
>> > org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1389)
>> > at
>> > org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:362)
>> > at
>> > org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1296)
>> > at
>> > org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:88)
>> > at
>> > org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:812)
>> > at
>> > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:557)
>> > at
>> > org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72)
>> > at
>> > org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:953)
>> > at
>> > org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:558)
>> > at
>> > org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:918)
>> > at
>> > org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:370)
>> > at
>> > org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1497)
>> > at
>> > org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1459)
>> > at
>> > org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:848)
>> > at
>> > org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:287)
>> > at
>> > org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545)
>> > at
>> > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
>> > at
>> > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
>> > at
>> > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
>> > at
>> > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
>> > at
>> > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
>> > at
>> > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
>> > at
>> > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108)
>> > at
>> > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
>> > at
>> > org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:403)
>> > at
>> > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
>> > at
>> > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
>> > at
>> > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
>> > at
>> > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
>> > at
>> > org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:167)
>> > at
>> > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
>> > at
>> > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
>> > at
>> > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
>> > at
>> > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
>> > at
>> > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
>> > at
>> > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
>> > at org.eclipse.jetty.server.Server.start(Server.java:419)
>> > at
>> > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108)
>> > at
>> > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
>> > at org.eclipse.jetty.server.Server.doStart(Server.java:386)
>> > at
>> > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
>> > at
>> > org.apache.nifi.web.server.JettyServer.start(JettyServer.java:935)
>> > at org.apache.nifi.NiFi.<init>(NiFi.java:158)
>> > at org.apache.nifi.NiFi.<init>(NiFi.java:72)
>> > at org.apache.nifi.NiFi.main(NiFi.java:297)
>> >
>> > On Wed, Aug 14, 2019 at 7:42 AM Bryan Bende <bbe...@gmail.com> wrote:
>> >>
>> >> Without access to the code for your NAR I can only really guess, but it
>> >> sounds like an exception is happening when trying to call the
>> >> constructor of your processor and then it bounces into creating a ghost
>> >> processor.
>> >>
>> >> What is in the logs at the time you get the ghost processor?
>> >>
>> >> On Tue, Aug 13, 2019 at 10:54 PM Bimal Mehta <bimal...@gmail.com> wrote:
>> >>>
>> >>> Does that mean I need to recreate the processor? Or there is some
>> >>> workaround?
>> >>>
>> >>> The processor gets unpacked and its bundled dependencies go in NAR_INF.
>> >>> However when I drag the processor on the canvas, it comes with a yellow
>> >>> triangle (and gives the error message I stated above) and properties are
>> >>> missing as well.
>> >>>
>> >>>
>> >>> On Tue, Aug 13, 2019 at 10:47 PM Bryan Bende <bbe...@gmail.com> wrote:
>> >>>>
>> >>>> I don’t remember all the reasoning behind the change, but it had to do
>> >>>> with an issue when we upgraded Jetty...
>> >>>>
>> >>>> https://issues.apache.org/jira/browse/NIFI-5479
>> >>>>
>> >>>> On Tue, Aug 13, 2019 at 9:47 PM Bimal Mehta <bimal...@gmail.com> wrote:
>> >>>>>
>> >>>>> Yes it does show as an option.
>> >>>>> One thing I noticed is that the when the nar is unpacked, the bundled
>> >>>>> dependencies are inside META_INF in the work folder in NiFi 1.6.0,
>> >>>>> however in NiFI 1.9.0 they go inside NAR_INF.
>> >>>>> Why does this happen?
>> >>>>> It seems the custom processor that we have uses Springboot, and
>> >>>>> references applicationcontext file which was inside META_INF when it
>> >>>>> was built. However I cant see that file anymore in the unpacked nar.
>> >>>>>
>> >>>>> On Tue, Aug 13, 2019 at 8:57 PM Bryan Bende <bbe...@gmail.com> wrote:
>> >>>>>>
>> >>>>>> Does that custom processor type show as an option if you try to add a
>> >>>>>> new processor to the canvas?
>> >>>>>>
>> >>>>>> On Tue, Aug 13, 2019 at 4:54 PM Bimal Mehta <bimal...@gmail.com>
>> >>>>>> wrote:
>> >>>>>>>
>> >>>>>>> Hi Mike and Bryan,
>> >>>>>>>
>> >>>>>>> One of my custom processors appears as inactive in NiFi with a
>> >>>>>>> yellow triangle error.
>> >>>>>>> When I hover over it I see a message saying 'Missing Processor'
>> >>>>>>> validated against 'Any Property' is invalid. This is not a valid
>> >>>>>>> processor.
>> >>>>>>> In the log it seems to invoke GhostProcessor.java which is giving
>> >>>>>>> the above error when restarting nifi.
>> >>>>>>> This custom processor sits (with my other processors) in my
>> >>>>>>> custom_lib folder and I have provided that path in the nifi
>> >>>>>>> properties file as
>> >>>>>>>
>> >>>>>>> nifi.nar.library.directory.custom=/opt/nifi/custom_lib
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Not sure what I missed?
>> >>>>>>>
>> >>>>>>> Do I need to make entry of this custom processor somewhere?
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Thu, Aug 8, 2019 at 9:14 AM Bimal Mehta <bimal...@gmail.com>
>> >>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>> Thanks Mike and Bryan.
>> >>>>>>>> Yes it seems my template was still referring the old version.
>> >>>>>>>> I will have it updated now and will reimport.
>> >>>>>>>> Also the version of NiFi we are using is the one that comes with
>> >>>>>>>> CDF. I am not sure if CDF supports 1.9.2 yet or not. I will reach
>> >>>>>>>> out to Cloudera and see if we can get it upgraded.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Thu, Aug 8, 2019, 8:51 AM Bryan Bende <bbe...@gmail.com> wrote:
>> >>>>>>>>>
>> >>>>>>>>> What is in the template for the bundle coordinates of your
>> >>>>>>>>> processor?
>> >>>>>>>>> and does that match the coordinates of the NAR that is deployed?
>> >>>>>>>>>
>> >>>>>>>>> Example:
>> >>>>>>>>>
>> >>>>>>>>> <bundle>
>> >>>>>>>>> <group>org.apache.nifi</group>
>> >>>>>>>>> <artifact>nifi-update-attribute-nar</artifact>
>> >>>>>>>>> <version>1.10.0-SNAPSHOT</version>
>> >>>>>>>>> </bundle>
>> >>>>>>>>>
>> >>>>>>>>> If you made a new version of your NAR, say 2.0.0 and your template
>> >>>>>>>>> references 1.0.0, then you'll need to update your template.
>> >>>>>>>>>
>> >>>>>>>>> On Wed, Aug 7, 2019 at 10:05 PM Mike Thomsen
>> >>>>>>>>> <mikerthom...@gmail.com> wrote:
>> >>>>>>>>> >
>> >>>>>>>>> > If it's happening immediately upon trying to import the
>> >>>>>>>>> > template, I believe that's the error message saying that the 1.9
>> >>>>>>>>> > instance cannot find the NAR file which provided the processor.
>> >>>>>>>>> > Also, if you're referring to 1.9.0 and not 1.9.2 you're going to
>> >>>>>>>>> > want to upgrade to the latter because there are a few critical
>> >>>>>>>>> > bugs fixed in 1.9.2.
>> >>>>>>>>> >
>> >>>>>>>>> > On Wed, Aug 7, 2019 at 9:19 PM Bimal Mehta <bimal...@gmail.com>
>> >>>>>>>>> > wrote:
>> >>>>>>>>> >>
>> >>>>>>>>> >> Thanks Bryan.
>> >>>>>>>>> >> My custom processors are part of a template. However when I try
>> >>>>>>>>> >> to import my template in NiFi 1.9, I get an error message saying
>> >>>>>>>>> >> PutFeedMetadata is not known to this NiFi instance. I did
>> >>>>>>>>> >> update all the dependencies to NiFi 1.9 and even the plugins.
>> >>>>>>>>> >> We are using a Cloudera distributed version of NiFi 1.9.
>> >>>>>>>>> >> Any idea why is this happening?
>> >>>>>>>>> >>
>> >>>>>>>>> >> Thanks
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >>
>> >>>>>>>>> >> On Wed, Aug 7, 2019 at 3:46 PM Bryan Bende <bbe...@gmail.com>
>> >>>>>>>>> >> wrote:
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Hello,
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> Most likely your processor built against 1.6 would run fine in
>> >>>>>>>>> >>> 1.9,
>> >>>>>>>>> >>> but to make sure you just need to update any nifi dependencies
>> >>>>>>>>> >>> in your
>> >>>>>>>>> >>> poms to 1.9.2.
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> If you created your project from the archetype and didn't
>> >>>>>>>>> >>> change
>> >>>>>>>>> >>> anything, then this should just be changing the parent in the
>> >>>>>>>>> >>> root pom
>> >>>>>>>>> >>> to the new version of nifi-nar-bundles.
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> If you set it up yourself, then anywhere you depend on
>> >>>>>>>>> >>> nifi-api you
>> >>>>>>>>> >>> need to change.
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> -Bryan
>> >>>>>>>>> >>>
>> >>>>>>>>> >>> On Wed, Aug 7, 2019 at 3:18 PM Bimal Mehta
>> >>>>>>>>> >>> <bimal...@gmail.com> wrote:
>> >>>>>>>>> >>> >
>> >>>>>>>>> >>> > Hi,
>> >>>>>>>>> >>> >
>> >>>>>>>>> >>> > If we have a custom processor that was created with NiFi
>> >>>>>>>>> >>> > 1.6, what are the steps we need to follow to make it work in
>> >>>>>>>>> >>> > 1.9?
>> >>>>>>>>> >>> > Is there some sort of steps that explains the jar and pom
>> >>>>>>>>> >>> > updates we need to do for making it work in 1.9?
>> >>>>>>
>> >>>>>> --
>> >>>>>> Sent from Gmail Mobile
>> >>>>
>> >>>> --
>> >>>> Sent from Gmail Mobile
>> >>
>> >> --
>> >> Sent from Gmail Mobile
