Cool - thanks for verifying that, Matt! https://issues.apache.org/jira/browse/NIFI-854
On Fri, Aug 14, 2015 at 11:43 AM, Matt Gilman <[email protected]> wrote: > Brian, > > I see the same behavior regarding Controller Service creation when > referenced by Processor's in sub Groups. Please file the JIRA ticket and it > will be addressed in our release. Thanks! > > Matt > > On Fri, Aug 14, 2015 at 11:30 AM, Brian Ghigiarelli <[email protected]> > wrote: > > > Matt and Bryan, > > > > Thanks for the detailed explanation. The class loading aspect makes > > perfect sense, and yes, importing as disabled is understood as well. > > > > I do think I've come across a small bug with automatically creating > > Controller Services on template imports though. NiFi is not > automatically > > creating Controller Services for Processors inside of Process Groups on > > template imports. Can one of you check me on this before we write up a > Jira > > ticket? > > > > Steps to reproduce: > > 1. Start fresh (clean canvas, no controller services) > > 2. Create Process Group and go into it > > 3. Add DetectDuplicate processor to Process Group > > 4. Configure DetectDuplicate processor with a new > > DistributedMapCacheClientService and enable that service so that the > > DetectDuplicate processor is valid and ready to run > > 5. Navigate to root canvas (see the Process Group) > > 6. Create template from root canvas (name: "Controller Service Test") > > 7. Delete everything from the canvas and disable + delete the controller > > service so that we're essentially back to the start > > 8. Drag and drop the "Controller Service Test" template onto the canvas > > (expecting that everything will be auto-created) > > 9. Navigate into the Process Group and notice that the DetectDuplicate > > processor is invalid because the DistributedMapCacheClientService ID > cannot > > be found > > > > As an additional test, I created a template with a processor at the > > top-level and one inside the Process Group that are using the same > > Controller Service, and that works just fine. My guess is that the > > top-level processor is used to create the Controller Service and the > > inner-processor picks it up by that same ID. > > > > I can provide a sample template in a Jira ticket if that would help. > > > > Thanks, > > Brian > > > > On Fri, Aug 14, 2015 at 8:24 AM, Bryan Bende <[email protected]> wrote: > > > > > Brian, > > > > > > Glad it worked! > > > > > > I think it was just a combination of things... I had previously spent a > > lot > > > of time studying the existing dependency setup in nifi-standard-bundle, > > as > > > well as the dev > > > guide section on NARs [1]. Also, someone else had ran into a similar > > > problem recently and it turned out to be a dependency problem as well. > > > > > > There should be a Maven archetype for Controller Services [2] coming > soon > > > which might help nail down the correct dependency setup, also after > > seeing > > > your email last night I put together a bare-bones project [3] that has > a > > > custom processor using a custom controller service (both created using > > > the archetypes) to serve as an example. > > > > > > -Bryan > > > > > > [1] > > https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#nars > > > [2] https://issues.apache.org/jira/browse/NIFI-736 > > > [3] https://github.com/bbende/nifi-dependency-example > > > > > > > > > > > > On Fri, Aug 14, 2015 at 8:17 AM, Matt Gilman <[email protected]> > > > wrote: > > > > > > > Brian, > > > > > > > > The NAR dependency stuff all breaks down to providing classpath > > isolation > > > > and not polluting the classpath of downstream NARs with dependencies > > > > required for Controller Service implementation. We try to keep > > Controller > > > > Service interfaces lean by not depending on external libs in the > > > interface > > > > (generally a good practice anyways). Both the implementation of the > > > > Controller Service and the any Processor NAR that want to consume the > > > > Controller Service needs to depend on the Controller Service API NAR > > > (where > > > > the interface lives). This established a common parent ClassLoader > > which > > > > allows the service to be access across NARs. If your building > Processor > > > > NARs that depend on custom Controller Services and standard > Controller > > > > Services, you'll want to have your custom Controller Services API NAR > > > > depend on the standard Controller Services API NAR. The NAR > dependency > > > tree > > > > (if you will) maps to a ClassLoader hierarchy within NiFi. > > > > > > > > Controller Services being added in a disabled state is remaining > > > consistent > > > > with Processors (and other components) in template. Also, because > > > sensitive > > > > properties are not included in templates, they may require the user > to > > > > complete the configuration prior to enabling. > > > > > > > > Matt > > > > > > > > On Fri, Aug 14, 2015 at 8:01 AM, Brian Ghigiarelli < > > [email protected]> > > > > wrote: > > > > > > > > > Bryan, > > > > > > > > > > Thank you!! Adding a NAR dependency to my standard-processors NAR > did > > > the > > > > > trick to get NiFi to recognize the service for these processors and > > > > > auto-create a new instance of the Controller Service on template > > > > import. I > > > > > owe ya a beer or six the next time our paths cross. > > > > > > > > > > How did you figure out the required NAR dependency? Any great > > debugging > > > > > tricks to share, or was it your existing depth of NiFi knowledge > that > > > led > > > > > you to that solution? > > > > > > > > > > As a follow-on to that, I'm seeing that Controller Services created > > via > > > > > Template instantiation are created in "Disabled" state. Is that > > > > expected? > > > > > Is there any way to drag-and-drop a template with controller > services > > > > onto > > > > > a canvas and have the newly created controller services enabled? > > > > > > > > > > Thanks again, > > > > > Brian > > > > > > > > > > On Thu, Aug 13, 2015 at 5:57 PM, Bryan Bende <[email protected]> > > wrote: > > > > > > > > > > > Brian, > > > > > > > > > > > > I just ran into this exact scenario today while testing a custom > > > > > controller > > > > > > service. > > > > > > > > > > > > The cause of the problem was that my custom service NAR, did not > > > have a > > > > > NAR > > > > > > dependency on the service API NAR because I had accidentally > marked > > > the > > > > > API > > > > > > NAR as provided in the pom file. Once I removed the provided > scope > > > > > > everything worked. > > > > > > > > > > > > You can check the MANIFEST file in your custom service NAR... > under > > > > your > > > > > > NIFI_HOME/work/nar/extensions/{your-nar}/META-INF/MANIFEST.MF it > > > should > > > > > > have a "Nar-Dependency-Id" on your API NAR. If it is missing > that, > > > that > > > > > is > > > > > > likely the problem. > > > > > > > > > > > > -Bryan > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Aug 13, 2015 at 5:22 PM, Brian Ghigiarelli < > > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > NiFi Gurus, > > > > > > > > > > > > > > Regarding Controller Services, I'm seeing some strange behavior > > > > > > regarding a > > > > > > > custom Controller Service only showing its ID in a Processor > and > > > was > > > > > > > wondering if anyone's encountered similar issues. Seeing this > on > > > the > > > > > > > latest develop branch, 0.3.0-SNAPSHOT. > > > > > > > > > > > > > > Steps taken: > > > > > > > 1. Create instance of Controller Service in the Controller > > > Settings > > > > > > > > Controller Services tab > > > > > > > 1a. Enable that instance of the Controller Service > > > > > > > 2. Add new Processor on canvas that will use Controller Service > > > > > > > 3. Configure Processor > > > > > > > 3a. Existing Controller Service does not show in property > > options. > > > > Only > > > > > > "No > > > > > > > value" and "Create new service..." > > > > > > > 3b. Clicking "Create new service..." opens modal with the > custom > > > > > > Controller > > > > > > > Service as the only item in the dropdown > > > > > > > 3c. Clicking "Create" button succeeds, but puts the UUID of the > > > > > > Controller > > > > > > > Service in the Value of the Processor Property. > > > > > > > > > > > > > > Observed state: > > > > > > > - Processor is still invalid > > > > > > > - The ID in the Processor Property matches the ID of the second > > > > > > (disabled) > > > > > > > instance of the Controller Service in the Controller Settings > > > modal. > > > > > > > > > > > > > > There are no errors in nifi-app.log - only contains log > messages > > > from > > > > > the > > > > > > > StandardControllerServiceProvider that it "Created Controller > > > Service > > > > > of > > > > > > > type [my-implementation-class] with identifier > > > > > > > [UUID-that-shows-in-the-GUI]" > > > > > > > > > > > > > > Adding a DetectDuplicate processor with the > > > > > > > DistributedMapCacheClientService works just fine, so it'd hard > to > > > > point > > > > > > > fingers at the framework. Any thoughts on where to begin > > debugging > > > > > > and/or > > > > > > > loggers to turn up would be greatly appreciated! > > > > > > > > > > > > > > Thanks, > > > > > > > Brian > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Brian Ghigiarelli > > > > > 570-878-9139 > > > > > > > > > > > > > > > > > > > > -- > > Brian Ghigiarelli > > 570-878-9139 > > > -- Brian Ghigiarelli 570-878-9139
