Matt, Mark, thanks for the great explanations! I'm learning a lot! :) So I went down the road described but I'm getting another error: groovy.lang.MissingMethodException: No signature of method: org.apache.nifi.lookup.script.ScriptedLookupService$1.getProperty() is applicable for argument types: (org.apache.nifi.components.PropertyDescriptor) values: [PropertyDescriptor[Database Connection Pool Services]]
Basically I declared a final static PropertyDescriptor DBCP_SERVICE and inside the initialize method I tried to get the DBCPService as outlined. Comparing my code to the QueryDatabaseTable processor I noticed when QDT grabs the DBCPService instance it's context is a ProcessContext while in my GroovyLookupClass's initialize method context is a ControllerServiceInitializationContext so it's seem I'm using the wrong object right? Where perform the call for context.getProperty(DBCP_SERVICE).asControllerService(DBCPService)? ( https://github.com/apache/nifi/blob/c10ff574c4602fe05f5d1dae5eb0b1bd24026c02/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/QueryDatabaseTable.java#L191 ) A minor notice for future reference by other users: it took me a while to get the PropertyDescriptor working because declaring it was not enough to make it shown at Properties dialog. I had to enable and than disable the ScriptedProcessor it at least once to have it shown (guessing the code was not executed). Thanks again for all the support. 2017-11-14 15:52 GMT-02:00 Matt Burgess <[email protected]>: > Mark, > > Good point, I forgot the ScriptedLookupService is itself a > ConfigurableComponent and can add its own properties. The original > script from my blog comes from ExecuteScript, where you can't define > your own properties. I was just trying to make that work instead of > thinking about the actual problem, d'oh! > > Eric, rather than trying to get at a DBCPConnectionPool defined > elsewhere, you can add a property from your ScriptedLookupService that > is itself a reference to DBCPService. Then the user will see in the > properties a dropdown list of DBCPConnectionPool instances, just like > the other processors that use them (ExecuteSQL, e.g.). Mark outlined > that approach, and it is definitely way better. Sorry for the wild > goose chase, although I guess it was only me that wasted my time :P > Guess it's time to add a new post using this technique instead! > > Thanks, > Matt > > > On Tue, Nov 14, 2017 at 12:43 PM, Mark Payne <[email protected]> wrote: > > Matt, Eric, > > > > The typical pattern that you would follow for obtaining a Controller > Service would be to > > return a property that uses the identifiesControllerService() method. > For example: > > > > static final PropertyDescriptor MYSQL_CONNECTION_POOL = new > PropertyDescriptor.Builder() > > .name("Connection Pool") > > .identifiesControllerService(DBCPService.class) > > .required(true) > > .build(); > > > > Then, to obtain that controller service, you would access it as: > > > > final DBCPService connectionPoolService = context.getProperty(MYSQL_ > CONNECTION_POOL).asControllerService(DBCPService.class) > > > > This allows the user to explicitly choose which controller service that > they want to use. > > > > Attempting to obtain a Controller Service by name will certainly cause > some problems, > > as you have already learned :) The API was not intended to work that > way, so you see > > that doing so can become difficult. There are a few reasons that we > don't want to retrieve the service > > by name: > > > > 1. It would require that the user know that they need to have a service > with > > that name. Then they would have to leave the configuration of your > service, and they would > > have to create a service with that name. Plus it would likely not be > obvious that they need to do this. > > 2. The framework would not know that your service is referencing the > connection pool service, so if > > the connection pool service is disabled, your service would still be > valid and the lifecycle management > > would not work as intended. > > 3. Controller Service names are not unique. So you may get the wrong one > if there are multiple with > > the same name. In fact, over 10 different iterations you could get 10 > different services instead of always > > getting the same service. > > > > So I guess the question is: Is there a reason that the typical approach > of identifying the service in a > > Property Descriptor doesn't work for your use case? > > > > Thanks > > -Mark > > > > > > > > > > > >> On Nov 14, 2017, at 12:11 PM, Matt Burgess <[email protected]> > wrote: > >> > >> Eric, > >> > >> So I just learned ALOT about the bowels of the context and > >> initialization framework while digging into this issue, and needless > >> to say we will need a better way of making this available to scripts. > >> Here's some info: > >> > >> 1) The ControllerServiceInitializationContext object passed into > >> initialize() is an anonymous object that passes along the > >> ScriptedLookupService's context objects, such as the > >> ControllerServiceLookup. > >> 2) The ControllerServiceLookup interface does not have a method > >> signature for getControllerServiceIdentifiers(Class, String) to pass > >> in the process group id. > >> 3) The ControllerServiceLookup object returned by the > >> ControllerServiceInitializationContext.getControllerServiceLookup() > >> method is a StandardControllerServiceInitializationContext > >> 4) Note that the context object passed into the initialize() method > >> and the one returned by context.getControllerServiceLookup() are > >> different (but both are ControllerServiceInitializationContext impls) > >> 5) The StandardControllerServiceInitializationContext object contains > >> a private ControllerServiceProvider called serviceProvider of type > >> StandardControllerServiceProvider, the anonymous context object does > >> not > >> 6) The StandardControllerServiceInitializationContext object delegates > >> the getControllerServiceIdentifiers(Class) method to the > >> serviceProvider > >> 7) serviceProvider (a StandardControllerServiceProvider) does not > >> allow the call to the getControllerServiceIdentifiers(Class) > >> signature, and instead throws the error you're seeing > >> 8) None of these objects can get at the process group ID. This is > >> because they are not associated with a ConfigurableComponent > >> 9) ScriptedLookupService, after calling the script's initialize() > >> method, will then call the script's onEnabled(ConfigurationContext) > >> method if it exists. This is currently undocumented [1] > >> 10) The script's onEnabled(ConfigurationContext) method will get a > >> StandardConfigurationContext object > >> 11) The StandardConfigurationContext object has a private > >> ConfiguredComponent named component, it is actually a > >> StandardControllerServiceNode object > >> 12) You can get the process group ID by calling the component's > >> getProcessGroupIdentifier() method > >> 13) The StandardConfigurationContext object also has a private > >> ControllerServiceLookup named serviceLookup, it is actually a > >> StandardControllerServiceProvider object > >> 14) Since we can get a process group ID from #11-12, we can now call > >> the supported method on the ControllerServiceProvider interface, > >> namely getControllerServiceIdentifiers(Class, String) > >> 15) Getting at private members (#11 &13) is allowed in Groovy, but > >> IIRC only works if you don't have a security manager/policies on the > >> JVM. > >> > >> TL;DR You can't currently get controller services by name in the > >> initialize() method, you have to implement onEnabled instead. If you > >> want to use logging, however, you'll need to save off the logger in > >> the initialize() method. Here's a working version of onEnabled: > >> > >> void onEnabled(ConfigurationContext context) { > >> lookup = context.serviceLookup > >> processGroupId = context.component?.processGroupIdentifier > >> /* Get sql-connection */ > >> def dbcpServiceId = > >> lookup.getControllerServiceIdentifiers(ControllerService, > >> processGroupId).find { > >> cs -> lookup.getControllerServiceName(cs) == 'MySQLConnectionPool' > >> } > >> def conn = lookup.getControllerService( > dbcpServiceId)?.getConnection() > >> } > >> > >> Hope this helps. I will think some more on how to make everything > >> fluid and legit -- Mark Payne, could use your help here :) > >> > >> Regards, > >> Matt > >> > >> On Tue, Nov 14, 2017 at 6:13 AM, Eric Chaves <[email protected]> wrote: > >>> Hi Folks, > >>> > >>> I need to get an instance of DBCPService inside my > ScriptedLookupService and > >>> for that I'm following Matt's post > >>> http://funnifi.blogspot.com.br/2016/04/sql-in-nifi-with- > executescript.html > >>> > >>> In my groovy class I've overrided the initialize method and performing > the > >>> lookup there but I'm getting the following error: > >>> > >>> java.lang.UnsupportedOperationException: Cannot obtain Controller > Service > >>> Identifiers for service type interface > >>> org.apache.nifi.controller.ControllerService without providing a > Process > >>> Group Identifier > >>> > >>> > >>> @Override > >>> void initialize(ControllerServiceInitializationContext context) > throws > >>> InitializationException { > >>> log = context.logger > >>> /* Get sql-connection */ > >>> def lookup = context.controllerServiceLookup > >>> def dbcpServiceId = > >>> lookup.getControllerServiceIdentifiers(ControllerService).find { > >>> cs -> lookup.getControllerServiceName(cs) == > 'MySQLConnectionPool' > >>> } > >>> conn = lookup.getControllerService(dbcpServiceId)?.getConnection() > >>> log.info("sql conn {}", conn) > >>> } > >>> > >>> Is there other way to find service identifiers? > >>> > >>> Regards, > > >
