Michael That is indeed strange. Quick question, is there any chance we can look at your code (the custom CS). Also, what version of NiFi you are using? Will be looking into it, but any additional info is helpful
Cheers Oleg > On Jan 10, 2017, at 1:14 PM, Knapp, Michael <[email protected]> > wrote: > > Devs, > > For some reason NIFI is not working with some custom controller services I > have written. I wrote new implementations of AWSCredentialsProviderService, > that aim to work with session tokens. I am hoping to run NIFI from my local > machine and to be able to connect with AWS using session tokens. > > For both implementations I tried, it fails when I try to create them from the > web UI. I created a PutS3Object processor, and configured the “AWS > Credentials Provider Service” property. From that property I tried to create > a new service and selected my custom implementation. When I click “create” > the value for the credentials provider service is the ID of the controller > service, not its name. While my controller services require several > properties to be set, the web UI is not letting me set them. Usually I see > an arrow next to the property, which allows me to configure a controller > service, but I am not getting that now. I looked in the nifi-app logs, and I > do not see any exception, I have even set the logging to TRACE for all > things, and still don’t see any problem in the logs. The PutS3Object > processor is not validating because the controller service is found to be > invalid. I tried creating a unit test, it seems to work for me in tests, but > I can’t use TestRunners because that is processor oriented, not meant for > controller services. I have a suspicion that spring’s aspect oriented > programming is somehow fuddling with my service. > > Does anybody know what I am doing wrong here? > > Other unrelated comments: > > 1. The first time you unpack NIFI it takes super long for it to start > for me, like a half hour or more. I think you should make it easy for people > to scale back their NIFI implementation. Really I would like to start it > with just the minimum NAR files for it to start, and I can add others that I > need. Maybe a sub-directory in lib for the essential nars could help people > separate the essential stuff from the optional nars. The first time I tried > installing it, I thought it was broken when really it just was taking forever > (over 30 minutes). I think that new users will probably abandon NIFI if they > can’t get it to start quickly out of the box. Maybe split the optional nars > into an “extra-lib”, and people can move those into lib as necessary for > their goals. > > 2. Building NIFI from source takes over an hour for me, really I just > want to build the bare minimum things to get it to start. I tried creating > maven profiles to have it build just the minimum pieces, but this proved to > be non-trivial as maven does not seem to respect the “modules” tag in > profiles, and the nifi-assembly project requires all of the optional nars to > also be built. Creating this might be too complicated for me. Has anybody > thought about supporting a quick/minimal build? > > 3. The “nifi-aws-processors” is challenging to use because in one > project they have defined the interfaces for controller services > (AWSCredentialsProviderService) and also included the services. I tried > creating my own nar with an implementation of AWSCredentialsProviderService, > but since it depended on “nifi-aws-processors”, my nar was also re-hosting > all of the AWS processors. I was facing a lot of classpath issues because of > this. I worked around this by using maven shade to shade in the > “nifi-aws-processors” into my own jar, but excluding the services it > provided. Then in my nar project I had to exclude the dependency on > “nifi-aws-processors”. This was a lot of work on my part when all they > needed to do was split that project into api, api-nar, impl, and impl-nar. > > 4. I think it is very confusing how there is a “Controller Services” > for the entire NIFI canvas, and separate ones for individual processor > groups. It seems that processors cannot use global controller services, and > I am still uncertain about why I would ever create a global one. From Nifi > settings, I would like to also see controller services in processor groups, > and vice versa. From a processor, I would like to assign controller services > that are global in scope, not limited to a processor group. I think this is > something that will confuse and frustrate a lot of new developers, driving > them to consider competing products. > > 5. I think the developer guide needs some clarification on what jars > are provided and not. New developers will be unsure if they should include > “nifi-api” as a provided or compile dependency, and same goes for > nifi-framework-core. > > 6. Perhaps the maven-nar-plugin could let people tell NIFI to only use > services listed under a certain set of bundled-dependencies. For example, my > code depends on “nifi-aws-processors”, but I don’t want my nar to also host > the services in that jar. From the MOJO you are able to exclude entire > artifacts, but you can’t exclude the services within certain artifacts. This > might be a problem to fix on the classloader side instead of from the maven > plugin, I don’t know which is easier. > > Please help, > > Michael Knapp > Capital One > ________________________________________________________ > > The information contained in this e-mail is confidential and/or proprietary > to Capital One and/or its affiliates and may only be used solely in > performance of work or services for Capital One. The information transmitted > herewith is intended only for use by the individual or entity to which it is > addressed. If the reader of this message is not the intended recipient, you > are hereby notified that any review, retransmission, dissemination, > distribution, copying or other use of, or taking of any action in reliance > upon this information is strictly prohibited. If you have received this > communication in error, please contact the sender and delete the material > from your computer.
