As a follow up, while implementing [1] I ran into [2] which is a larger issue but affects the RecordSetWriters more than Readers as they add more properties that are expected to come (in our case) from the script. That makes things a bit more complicated, but I am still looking into it. If you are coding a ScriptedReader you might be ok with the guidance I gave earlier, but I have my doubts that trying to use a ScriptedRecordSetWriter with a SchemaRegistry will work right now. I'll post any findings in either/both [1] and [2], and as always I welcome all comments, questions, and suggestions :)
Regards, Matt [1] https://issues.apache.org/jira/browse/NIFI-7343 [2] https://issues.apache.org/jira/browse/NIFI-5115 On Wed, Apr 8, 2020 at 8:19 PM Matt Burgess <[email protected]> wrote: > > Jairo, > We should probably move this to the dev list since we're getting into > the NiFi API, but I wasn't sure you were subscribed, if you are let me > know and we can move it over there. > > StandardConfigurationContext is in framework-core so you should not > count on being able to use that class per se. Instead you should add a > property like the one in SchemaAccessUtils: > > public static final PropertyDescriptor SCHEMA_REGISTRY = new > PropertyDescriptor.Builder() > .name("schema-registry") > .displayName("Schema Registry") > .description("Specifies the Controller Service to use for the > Schema Registry") > .identifiesControllerService(SchemaRegistry.class) > .required(false) > .build(); > > Then from your code you can add that property in your overridden > getSupportedPropertyDescriptors() method, and when you need it you can > get the property from the context, but don't refer to it as a > StandardConfigurationContext, there's an interface that has the same > methods and is in the API that you have access to already: > > final SchemaRegistry schemaRegistry = > context.getProperty(SCHEMA_REGISTRY).asControllerService(SchemaRegistry.class) > > Note that we're not getting the registry directly from the context, > rather we're getting the property value from the context and then > finding the corresponding SchemaRegistry, I believe that's done "under > the hood" by the ControllerServiceLookup, the > ".asControllerService(SchemaRegistry.class)" is the key part there. > > From there you can use the SchemaRegistry interface methods to get at > the other things, for example: > > descriptor = getPropertyDescriptor(SCHEMA_ACCESS_STRATEGY.getName()); > schemaAccess = context.getProperty(descriptor).getValue(); > schemaAccessStrategy = getSchemaAccessStrategy(schemaAccess, > schemaRegistry, context); > > Note that last method getSchemaAccessStrategy() is from > SchemaAccessUtils which you don't currently have access to, so you > might want to just copy all that over to your script. If you follow > the calls and code from SchemaRegistryService, you should be able to > bring any/all of that into your ScriptedReader so it can do the same > kind of processing. > > Regards, > Matt > > On Wed, Apr 8, 2020 at 7:03 PM Jairo Henao <[email protected]> wrote: > > > > Thanks Matt, > > > > I have made some changes using AvroReader as an example, but I am still > > able to access other ControllerServices. > > > > From the RecordReaderFactory I have access to a context that is of type > > org.apache.nifi.controller.service.StandardConfigurationContext but with > > this I don't know how to access the AvroSchemaRegistry. > > > > Do you know how I can access the ControllerServiceLookup? > > > > > > > > > > > > > > On Wed, Apr 8, 2020 at 3:19 PM Matt Burgess <[email protected]> wrote: > >> > >> Jairo, > >> > >> It is possible for a ScriptedReader to use a SchemaRegistry, although > >> it isn't currently as easy as it could be. When the Scripted > >> Reader/Writer components were added [1], much of the reusable code for > >> schema registry processing was in the service implementation NAR, so > >> the aforementioned Jira explains that using the SchemaRegistry > >> interface is certainly possible in ScriptedReader but fairly > >> cumbersome. Since then the SchemaRegistryService and associated util > >> classes were moved to a nifi-avro-record-utils [2] so > >> extensions/components can leverage schema registries. Unfortunately > >> nifi-avro-record-utils is not currently included in the scripting NAR, > >> but I have written up [3] to add this. > >> > >> In the meantime, take a look at SchemaRegistryService, > >> SchemaAccessUtils, and an implementation class such as AvroReader, you > >> could copy and paste all the necessary code to get your ScriptedReader > >> to interact with a schema registry. > >> > >> Regards, > >> Matt > >> > >> [1] https://issues.apache.org/jira/browse/NIFI-3734 > >> [2] https://issues.apache.org/jira/browse/NIFI-5123 > >> [3] https://issues.apache.org/jira/browse/NIFI-7343 > >> > >> On Wed, Apr 8, 2020 at 2:01 PM Jairo Henao <[email protected]> > >> wrote: > >> > > >> > Hi all, > >> > > >> > Is there a way from a ScriptedReader (Controller Service) to read a > >> > schema that is stored in an AvroSchemaRegistry? > >> > > >> > I am using Groovy as a language. > >> > > >> > Additionally, how can I read additional properties that I have added to > >> > my ControllerService from Groovy code. > >> > > >> > Thanks in advance > >> > > >> > > >> > -- > >> > Saludos > >> > > >> > Jairo Henao > > > > > > > > -- > > Saludos > > > > Jairo Henao > > > > Chat Skype: jairo.henao.05 > >
