James This article should help you get started https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions especially since you only have to go through the first part only “Maven Processor Archetype” (I am assuming you are familiar with maven). Also, make sure that you pick the right NiFi version when executing 'mvn archetype. . .’ command. The current website example use 1.0.0. Change it to 0.6.1 (see below).
$> mvn archetype:generate -DarchetypeGroupId=org.apache.nifi -DarchetypeArtifactId=nifi-processor-bundle-archetype -DarchetypeVersion=0.6.1 -DnifiVersion=0.6.1 After that you may want to copy the UnpackContent into some package within he bundle, rename it, adjust the class path to ensure it compiles and build. Make sure you add/modify your processor’s name in src/main/resources/META-INF/services/org.apache.nifi.processor.Processor file. Iet me know if you run into any issues. Cheers Oleg On Dec 14, 2016, at 3:08 PM, James McMahon <[email protected]<mailto:[email protected]>> wrote: I've never tried to create and deploy my own custom bundle but would like to try #3 as you recommend Oleg. If you can offer guidance to that process let me try that. Thank you. -Jim On Wed, Dec 14, 2016 at 2:57 PM, Oleg Zhurakousky <[email protected]<mailto:[email protected]>> wrote: James Not sure what kind of liberties you have with your environment, but here is what you can do outside of full upgrade. 1. You can download and drop the bundle (NAR) from NiFi 0.7 release into 0.6 release. That should work (just replace it in the ‘lib’ directory of NiFi home). However, this particular component is located in ’nifi-standard-nar’ which means it contains majority of the processors and controller services that come with NiFi. So while upgrading one component you will essentially upgrade almost all and I am not sure what it may lead to, but you can definitely try. 2. You can certainly checkout the NiFi 0.6.1 source from the repo and apply the same patch on UpdateContent, build NAR and replace it as described in the step above, Even though you’ll still be replacing the entire NAR the changes are only going to be in that single component. 3. You can create your own custom bundle (NAR), copy the UnpackContent code from 0.7 branch, rename the actual class to something different (e.g., UnzipContent) and then deploy it as your own custom bundle and use it in your flow. 4. You can also apply patch through fixing the byte code in flight but that would require you to write javaagent and be familiar with byte code manipulations frameworks. Not advisable. . . Honestly, aside from full upgrade the only viable option here is #3 since it gives you full control and complete isolation from every other component in NiFi. Let me know what you think so we can guide you thru the process. Cheers Oleg On Dec 14, 2016, at 2:18 PM, James McMahon <[email protected]<mailto:[email protected]>> wrote: Oleg, I am bound to NiFi 0.6.1 by enterprise application dependencies. So this fix will not be in my baseline if I understand you correctly. Let me ask you this: is there any way I can build this into my code baseline - either through a code mod and rebuild or as a custom plugin feature specific to my flow? Thanks very much for your help. On Wed, Dec 14, 2016 at 10:59 AM, Oleg Zhurakousky <[email protected]<mailto:[email protected]>> wrote: James Could you also let us know what version of NiFi you are using? The issue with properly handling InvalidPathException was fixed in NiFi 0.7.0 as part of https://issues.apache.org/jira/browse/NIFI-920 It essentially has this catch block: } catch (final ProcessException | InvalidPathException e) { logger.error("Unable to unpack {} due to {}; routing to failure", new Object[]{flowFile, e}); session.transfer(flowFile, REL_FAILURE); So I am wondering if you are on the proviso release? Cheers Oleg On Dec 14, 2016, at 10:49 AM, Joe Witt <[email protected]<mailto:[email protected]>> wrote: James, Can you please share the full log entry for that failure. It is possible the processor is not catching the exception properly and routing the data to failure. It might simply be letting the exception loose thus the framework detects the issue and rollsback the session and yields the processor. Likely an easy thing to fix in the processor but please do share the full nifi-app.log entry for this. Thanks Joe On Wed, Dec 14, 2016 at 10:47 AM, James McMahon <[email protected]<mailto:[email protected]>> wrote: Hello. Am wondering if anyone knows how to overcome a challenge with unmappable file characters? I have used a GetFile processor to bring a large number of zip files into a NiFi flow. All read in successfully. I try to then use an UnpackContent processor to unzip the files to their individual file members. Most work just fine. However there appears to be a file that throws this error in UnpackContent: failed to process session due to java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters My processing stalls. Nothing else flows. What is the proper way to configure the UnpackContent processor step so that it shuttle such files off to the side when it encounters them, and permits the other files waiting in queue to process? I do now have a "failure" path established for my UnpackContent processor, but for some reason it does not send these problem files down that path. I suspect it may be because the zip files does unpack successfully but the underlying file(s) within the zip cause processing to choke. How can I engineer a flow to overcome this challenge? Thanks in advance for your help.
