The dependencies can definitely be confusing to wrap your head around. A good rule of thumb is that you would generally not have a direct jar dependency on anything under nifi-nar-bundles in the source tree. This would mean no jar dependencies on processors, controller services, reporting tasks etc. Now there are cases where a NAR can depend on another NAR, this is common when a processor needs to use a controller service, a good description of that is here [1], another example is how we have the hadoop-libraries-nar so multiple NARs can depend on the same set of hadoop libraries and not have to duplicate all those jars. Any other dependencies like things from nifi-commons, such as processor utils and others, are fair game to include in your NAR. If you look at the nifi lib directory you can see the following jars:
jcl-over-slf4j-1.7.12.jar jul-to-slf4j-1.7.12.jar log4j-over-slf4j-1.7.12.jar logback-classic-1.1.3.jar logback-core-1.1.3.jar nifi-api-0.4.0-SNAPSHOT.jar nifi-documentation-0.4.0-SNAPSHOT.jar nifi-nar-utils-0.4.0-SNAPSHOT.jar nifi-properties-0.4.0-SNAPSHOT.jar nifi-runtime-0.4.0-SNAPSHOT.jar slf4j-api-1.7.12.jar These are automatically available to every nar, anything else needs to be brought in by bundling the jars in your NAR, or by depending on another NAR. As for logging, nifi itself uses slf4j and logback, so the logger you get from getLogger() would be controlled from the logback.xml in the conf directory. I think for non-processor classes if you use the slf4j api that will be your best bet as I believe it will automatically use the same logback configuration, but others could correct me if I am wrong here. I believe it is also possible to use log4j directly with in your NAR, for example I know some third party client libraries use log4j and by having a dependency on log4j-over-slf4j it can somehow route all of the log4j calls back through the main slf4j configuration. Hope I didn't confuse things more here, let us know. [1] https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions#MavenProjectsforExtensions-LinkingProcessorsandControllerServices On Fri, Nov 20, 2015 at 1:26 AM, Mark Petronic <markpetro...@gmail.com> wrote: > Thanks so much Bryan. It is running now. It is starting to come together > but I'm still a little unclear on when to include nifi files to pick up > dependencies and when to directly include the upstream dependencies. Like > say some nifi nar like nifi-processor-utils already has a dependency on > commons-lang3 and my custom processor needs commons-lang3, should I satisfy > that by depending on nifi-processor-utils (which in know will already be in > the install) or should I add a dependency on commons-lang3 in my nar and > duplicate the libraries? I realize that there are version issues to be > concerned with which I believe is the genesis for nars to start with but > assume I need the same version, too? What't the general best practice > there? > > Also, now that I was able to run my processor, I have a question about > logging. I see that the processors use getLogger() to get a logger. But is > see in other places a more traditional use of LoggerFactory.getLogger(). > What is the best approach to logging from helper classes that I use in my > processor that are not inner classes of the processor class by defined > outside in separate class files because I intend to reuse them across > multiple processor types? I need to debug an error that I am seeing in one > of those helper classes and need to add some logging. > > Mark > > On Thu, Nov 19, 2015 at 8:23 PM, Bryan Bende <bbe...@gmail.com> wrote: > > > Hi Mark, > > > > Glad to hear you were able to get started building processors, and glad > > that blog post helped! > > > > I pulled down your code and built and deployed it. It looks like the > issue > > is that your processors pom has a dependency on: > > <dependency> > > <groupId>org.apache.nifi</groupId> > > <artifactId>nifi-standard-processors</artifactId> > > <version>0.4.0-SNAPSHOT</version> > > </dependency> > > > > Which means your NAR ends up having the standard processors jar in it, > but > > they are also deployed in nifi-standard-nar which causes some problems. > You > > can see the jar is there by looking > > in > > > work/nar/extensions/nifi-bigdata-nar-1.0.nar-unpacked/META-INF/bundled-dependencies/. > > > > I removed that dependency from the pom and it looked like the only > > compilation errors were on some missing json related libraries (jackson > and > > json path). I added these to your processor pom and it seems to deploy > now: > > > > <dependency> > > <groupId>com.jayway.jsonpath</groupId> > > <artifactId>json-path</artifactId> > > <version>2.0.0</version> > > </dependency> > > <dependency> > > <groupId>org.codehaus.jackson</groupId> > > <artifactId>jackson-mapper-asl</artifactId> > > <version>1.9.13</version> > > </dependency> > > > > Hope that helps. Let us know if you have any other questions! > > > > -Bryan > > > > > > On Thu, Nov 19, 2015 at 7:53 PM, Mark Petronic <markpetro...@gmail.com> > > wrote: > > > > > Well, I finally built my first processor. Was a nice experience. Nice > > APIs! > > > This was my first work with Maven and Nifi so, if things look > > > wrong/strange, please go easy on me. :) The issue is when I copy the > NAR > > > file in the lib directory and restart Nifi, it does not start up. I > tried > > > removing the work directory for a clean start, no joy. If I remove my > > NAR, > > > works fine. Wondering if one of you experts would not mind helping me > > over > > > this hump because I have no clue what is happening based on the error > > > message in the log. Seems like some other processor, DetectDuplicate, > is > > > failing to startup when mine is in there?? I pasted the startup log > > traces > > > here: http://pastebin.com/raw.php?i=UzraAE53. My processor code is > here: > > > https://bitbucket.org/mpetronic/nificustomprocessors. I would be very > > > grateful for any guidance. It seems like a pretty simple processor so I > > was > > > surprised this happened. The code is commented so hope that helps you > > > easily understand what is going on there. Oh, I started from this very > > > helpful post (thanks Bryan): > > > > > > > > > http://bryanbende.com/development/2015/02/04/custom-processors-for-apache-nifi/ > > > . > > > I am running a build of Nifi from the GitHub mirror, > > > commit 90f6830003b76204e25dec9fafec2488c8bda550. > > > > > > Thanks, > > > Mark > > > > > >
