Re: Custom Processor Upgrade

2019-08-20 Thread Bimal Mehta
Thanks all for the support. We resolved the issue by creating a new nar file itself. It seems the old version code was using some outdated dependencies which was not getting fixed when upgrading the processor. On Thu, Aug 15, 2019 at 2:06 PM James Srinivasan wrote: > I find strace (or procmon

Re: Custom Processor Upgrade

2019-08-15 Thread James Srinivasan
I find strace (or procmon for Windows) very handy to debug such resource loading issues. On Thu, 15 Aug 2019, 19:02 Bryan Bende, wrote: > I was making sure you didn't have any code that was dependent on the > internal structure of how the NARs are unpacked. > > I can't explain why it can't find

Re: Custom Processor Upgrade

2019-08-15 Thread Bryan Bende
I was making sure you didn't have any code that was dependent on the internal structure of how the NARs are unpacked. I can't explain why it can't find the application-context.xml since I don't have access to your code, but I don't see why that would be related to moving to NAR_INF from META_INF,

Re: Custom Processor Upgrade

2019-08-15 Thread Bimal Mehta
Its inside one of the jars within the NAR_INF folder. Does it need to be somehwere else? Also I think we extended the AbstractNiFiProcessor from the custom kylo processor while migrating it as kylo processor was not working as is in our environment.. Will check that and have it packaged in the

Re: Custom Processor Upgrade

2019-08-14 Thread Bryan Bende
Where is application-context.xml in your NAR? And how are you trying to load it in com.thinkbiganalytics.nifi.processor.AbstractNiFiProcessor ? I would expect it to be packaged into the jar that contains your processors, most likely in src/main/resources of the processors module which then ends

Re: Custom Processor Upgrade

2019-08-14 Thread Bimal Mehta
Ahh, seems like a Springboot error. Is it to do with upgraded Jetty server ? Caused by: org.springframework.beans.factory.BeanDefinitionStoreException: Unexpected exception parsing XML document from class path resource [application-context.xml]; nested exception is

Re: Custom Processor Upgrade

2019-08-14 Thread Bryan Bende
You have to add another instance of the processor which should generate the same stracktrace you sent earlier, except this time there should be a second part to it with "Caused by " and then more of the stacktrace that wasn't there before. On Wed, Aug 14, 2019 at 4:41 PM Bimal Mehta wrote: >

Re: Custom Processor Upgrade

2019-08-14 Thread Bimal Mehta
Hi Bryan, I did what you said. This is what I got 2019-08-14 20:16:18,948 DEBUG [Validate Components Thread-3] o.a.n.controller.AbstractComponentNode Computed validation errors with Validation Context StandardValidationContext[componentId= 6fbe2407-7799-3908-f4c4-bf2f8940bf1e ,

Re: Custom Processor Upgrade

2019-08-14 Thread Bryan Bende
Can you edit logback.xml and add the following, the get the stacktrace again? This should include a root cause exception which we are missing right now. I think it takes about 20-30 seconds for logback to pick up the edits to logback.xml. On Wed, Aug 14, 2019 at 12:53 PM Bimal Mehta wrote: >

Re: Custom Processor Upgrade

2019-08-14 Thread Bimal Mehta
For the custom processor we have, we are extending the AbstractNifiProcessor.java. The processor is used to scan metadata of an incoming flow file. The error we get in logs is as below. 2019-08-13 23:21:21,529 ERROR [main] o.a.nifi.controller.ExtensionBuilder Could not create Processor of type

Re: Custom Processor Upgrade

2019-08-14 Thread Bryan Bende
Without access to the code for your NAR I can only really guess, but it sounds like an exception is happening when trying to call the constructor of your processor and then it bounces into creating a ghost processor. What is in the logs at the time you get the ghost processor? On Tue, Aug 13,

Re: Custom Processor Upgrade

2019-08-13 Thread Bimal Mehta
Does that mean I need to recreate the processor? Or there is some workaround? The processor gets unpacked and its bundled dependencies go in NAR_INF. However when I drag the processor on the canvas, it comes with a yellow triangle (and gives the error message I stated above) and properties are

Re: Custom Processor Upgrade

2019-08-13 Thread Bryan Bende
I don’t remember all the reasoning behind the change, but it had to do with an issue when we upgraded Jetty... https://issues.apache.org/jira/browse/NIFI-5479 On Tue, Aug 13, 2019 at 9:47 PM Bimal Mehta wrote: > Yes it does show as an option. > One thing I noticed is that the when the nar is

Re: Custom Processor Upgrade

2019-08-13 Thread Bimal Mehta
Yes it does show as an option. One thing I noticed is that the when the nar is unpacked, the bundled dependencies are inside META_INF in the work folder in NiFi 1.6.0, however in NiFI 1.9.0 they go inside NAR_INF. Why does this happen? It seems the custom processor that we have uses Springboot,

Re: Custom Processor Upgrade

2019-08-13 Thread Bryan Bende
Does that custom processor type show as an option if you try to add a new processor to the canvas? On Tue, Aug 13, 2019 at 4:54 PM Bimal Mehta wrote: > Hi Mike and Bryan, > > One of my custom processors appears as inactive in NiFi with a yellow > triangle error. > When I hover over it I see a

Re: Custom Processor Upgrade

2019-08-13 Thread Bimal Mehta
Hi Mike and Bryan, One of my custom processors appears as inactive in NiFi with a yellow triangle error. When I hover over it I see a message saying 'Missing Processor' validated against 'Any Property' is invalid. This is not a valid processor. In the log it seems to invoke GhostProcessor.java

Re: Custom Processor Upgrade

2019-08-08 Thread Bimal Mehta
Thanks Mike and Bryan. Yes it seems my template was still referring the old version. I will have it updated now and will reimport. Also the version of NiFi we are using is the one that comes with CDF. I am not sure if CDF supports 1.9.2 yet or not. I will reach out to Cloudera and see if we can

Re: Custom Processor Upgrade

2019-08-08 Thread Bryan Bende
What is in the template for the bundle coordinates of your processor? and does that match the coordinates of the NAR that is deployed? Example: org.apache.nifi nifi-update-attribute-nar 1.10.0-SNAPSHOT If you made a new version of your NAR, say

Re: Custom Processor Upgrade

2019-08-07 Thread Mike Thomsen
If it's happening immediately upon trying to import the template, I believe that's the error message saying that the 1.9 instance cannot find the NAR file which provided the processor. Also, if you're referring to 1.9.0 and not 1.9.2 you're going to want to upgrade to the latter because there are

Re: Custom Processor Upgrade

2019-08-07 Thread Bimal Mehta
Thanks Bryan. My custom processors are part of a template. However when I try to import my template in NiFi 1.9, I get an error message saying PutFeedMetadata is not known to this NiFi instance. I did update all the dependencies to NiFi 1.9 and even the plugins. We are using a Cloudera distributed

Re: Custom Processor Upgrade

2019-08-07 Thread Bryan Bende
Hello, Most likely your processor built against 1.6 would run fine in 1.9, but to make sure you just need to update any nifi dependencies in your poms to 1.9.2. If you created your project from the archetype and didn't change anything, then this should just be changing the parent in the root pom

Custom Processor Upgrade

2019-08-07 Thread Bimal Mehta
Hi, If we have a custom processor that was created with NiFi 1.6, what are the steps we need to follow to make it work in 1.9? Is there some sort of steps that explains the jar and pom updates we need to do for making it work in 1.9?